This guide covers how to use KokoroSharp with GPU acceleration for faster inference.
KokoroSharp offers several packages:
| Package | Platform | GPU Support | Requirements |
|---|---|---|---|
| KokoroSharp | All platforms | Runtime-dependant | Expects devs to bring their own runtime |
| KokoroSharp.CPU | All Platforms | CPU-ONLY | Plug & Play with just CPU |
| KokoroSharp.GPU | Windows, Linux, Mac (Intel) | NVIDIA (CUDA) | CUDA Toolkit + cuDNN |
| KokoroSharp.GPU.Windows | Windows | NVIDIA (CUDA) | CUDA Toolkit + cuDNN |
| KokoroSharp.GPU.Linux | Linux | NVIDIA (CUDA) | CUDA Toolkit + cuDNN |
The CPU runtime just works, and should be quick for real-time speech synthesis, even on old CPUs:
- Install KokoroSharp.CPU via NuGet, then run:
KokoroTTS tts = KokoroTTS.LoadModel();
tts.SpeakFast("Hello from GPU!", KokoroVoiceManager.GetVoice("af_heart"));For NVIDIA GPUs with CUDA support:
- Choose either KokoroSharp.GPU.Windows, KokoroSharp.GPU.Linux, or KokoroSharp.GPU.
- Download & install CUDA Toolkit (v12.x), and make sure it's in SYSTEM PATH.
- Download & install cuDNN, and make sure it's in SYSTEM PATH.
- Restart your IDE/terminal after installation, then, run:
var options = new SessionOptions();
options.AppendExecutionProvider_CUDA();
KokoroTTS tts = KokoroTTS.LoadModel(sessionOptions: options);
tts.SpeakFast("Hello from GPU!", KokoroVoiceManager.GetVoice("af_heart"));**IF you choose to NOT install CUDA/cuDNN, this is the full list of libraries needed to be loaded** [Click to expand]
- cudnn_engines_runtime_compiled64_9.dll- cudnn_engines_precompiled64_9.dll
- cudnn_heuristic64_9.dll
- cudnn_graph64_9.dll
- cublasLt64_12.dll
- cudnn_adv64_9.dll
- cudnn_ops64_9.dll
- cublas64_12.dll
- cudart64_12.dll
- cufft64_11.dll
- cudnn64_9.dll
I currently haven't managed to make KokoroSharp with CoreML. ONNX Supports it via options.AppendExecutionProvider_CoreML();, but there's no package that brings the runtime.
The standard workflow would be:
- Install KokoroSharp via NuGet.
- Build and install CoreML ONNX runtime, then run:
var options = new SessionOptions();
options.AppendExecutionProvider_CoreML();
KokoroTTS tts = KokoroTTS.LoadModel(sessionOptions: options);
tts.SpeakFast("Hello from GPU!", KokoroVoiceManager.GetVoice("af_heart"));If you manage to make KokoroSharp work with CoreML, please consider contributing to this file.
If you're getting an EntryPointNotFoundException, this means that you don't have the correct KokoroSharp package installed.
It's also possible to receive this error if you have more packages than you should. Deleting excess runtimes should resolve this.
DllNotFoundException means you have no runtime AT ALL. For plug & play support (still fast), use the KokoroSharp.CPU package.
Always reset your IDE/terminal after installing dependencies, to make sure the system path is in sync.