CosyVoice on Mac: CosyVoice 2 and 3 Setup Guide

Short answer: CosyVoice is an open source text to speech family from researchers at Alibaba Group, built around a language model plus a flow matching decoder, with zero-shot voice cloning from a reference clip. Murmur does not run CosyVoice. On a Mac you run it yourself from the official repository, and the published install path assumes Linux with Python 3.10, so plan on CPU inference or community patches before you get Apple GPU acceleration.

That distinction matters because search results mix three different things: the research papers, the GitHub repository, and the hosted demos. The papers describe model behavior. The repository describes one install recipe. Neither one promises a supported macOS build. This guide sticks to what the project's own README, model cards, papers, and issue tracker say, and it marks the places where Mac users are on their own.

What CosyVoice is, and why Mac support is a separate question

The project now lives under the QwenAudio/CosyVoice repository (the older FunAudioLLM links redirect there) and is licensed Apache-2.0. The architecture splits the job in two: a text to speech language model predicts supervised semantic speech tokens, then a conditional flow matching model turns those tokens into a Mel spectrogram for a given speaker, and a vocoder produces the waveform. The CosyVoice 2 paper describes a gradual semantic decoding procedure and says the design supports both streaming and non-streaming synthesis in one model.

The app ships a different catalog: Kokoro (bundled), Chatterbox Turbo, Chatterbox Multilingual, Qwen3-TTS Base, Voice Design, SparkTTS, Fish Audio S2 Pro, Breeze TTS 2, and OmniVoice. CosyVoice is not among them, and no setting in the app will make it appear. If you want CosyVoice behavior specifically, you need the Python project. If you want a Mac app that handles scripts, speakers, takes, and export without a terminal, compare the shipped models first in the local TTS model guide.

One more piece of small print: the README ends with a disclaimer that the content is for academic purposes and demonstrates technical capabilities. The repository license is still Apache-2.0, and both Hugging Face model cards list apache-2.0, but read the license and the card yourself before you build a commercial product on the weights.

CosyVoice 1, 2, and 3: what the official sources say changed

The CosyVoice 3 paper says the language model was scaled from 0.5B to 1.5B parameters and the flow matching model from 100M to 300M, and that this larger capacity improved its multilingual benchmark. The checkpoint the README actually tells you to download is Fun-CosyVoice3-0.5B-2512, which is 0.5B. Both facts can be true at once: the paper reports experiments at two sizes, and the released default is the smaller one. If you see a claim that CosyVoice 3 on your Mac is a 1.5B model, check which weights that person actually loaded.

The version jump also changes the language story. The CosyVoice 2 paper reports training data in Chinese, English, Japanese, and Korean, and its limitations section says the model supports only a limited number of languages. The CosyVoice 3 README lists nine languages (Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian) plus 18 or more Chinese dialects and accents, and both the README and the paper describe cross-lingual zero-shot cloning. The paper's data section counts 19 Chinese accents or dialects while the abstract says 18, so treat the dialect count as approximate and check the current card.

Zero-shot cloning: a reference clip plus its transcript

Zero-shot here means the model is not fine-tuned on your speaker. You supply a short reference recording and the matching text, and the model imitates the timbre, prosody, and style of that reference. In the CosyVoice 2 model card, the zero-shot call takes the text to synthesize, the prompt text, and the prompt wav path. A separate cross-lingual call takes the target text and the same prompt wav, and an instruct call takes a natural language instruction as a second argument after the text.

cosyvoice = AutoModel(model_dir='pretrained_models/CosyVoice2-0.5B') for i, j in enumerate(cosyvoice.inference_zero_shot('收到好友从远方寄来的生日礼物，那份意外的惊喜与深深的祝福让我心中充满了甜蜜的快乐，笑容如花儿般绽放。', '希望你以后能够做的比我还好呦。', './asset/zero_shot_prompt.wav')): torchaudio.save('zero_shot_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)

CosyVoice2-0.5B model card, zero-shot usage example

The reference transcript is not decoration. CosyVoice 2's in-context learning mode uses prompt text and prompt speech tokens together, and the card's own examples pass the prompt text explicitly. The CosyVoice 3 card adds an instruction prefix such as "You are a helpful assistant." before the prompt text. If your reference clip is noisy, clipped, or in a different language from its transcript, the clone will reflect that.

Only clone voices you own or have written permission to use. A local model removes the upload step, not the consent requirement. For a broader workflow around reference recordings and takes, see the voice cloning guide.

The streaming latency claim, with its context

The CosyVoice 3 README says the system supports text-in and audio-out streaming and "achieves latency as low as 150ms while maintaining high-quality audio output." That is the project's claim about its own streaming path. It is not a macOS benchmark, it does not say which hardware produced the figure, and it does not say how much text must arrive before the first audio chunk. Treat 150 ms as a best-case target to reproduce, not a number you should expect on an M1 laptop running CPU inference.

The CosyVoice 2 paper is more explicit about what the latency depends on. Its first-package latency is modeled as the chunk size M multiplied by the per-token time for the language model, the flow matching model, and the vocoder, summed together. Change the chunk size or the hardware and the number changes. The paper also reports that its streaming mode was nearly lossless compared with offline synthesis in typical tests, with a slight content consistency drop in hard cases. Those are paper results, not a measurement of your machine.

If you care about streaming behavior, measure it. The streaming latency measurement guide covers first-chunk timing and real-time factor on a Mac. A file-based editor is a different product category from a live streaming endpoint, so do not read this section as a claim about any desktop app.

Installing on macOS: the official steps and the gaps

The README's install section is short and Linux-oriented. These are its exact commands, minus the mirror flags the project adds for users in China:

git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git cd CosyVoice git submodule update --init --recursive conda create -n cosyvoice -y python=3.10 conda activate cosyvoice pip install -r requirements.txt

CosyVoice README, clone and install section, checked 2026-10-19

The submodule step is not optional. The FAQ says a missing Matcha-TTS directory produces a ModuleNotFoundError, and that you should run git submodule update --init --recursive if third_party/Matcha-TTS is absent. The requirements file pins torch 2.3.1 and torchaudio 2.3.1, and it installs onnxruntime for darwin, so a macOS CPU install is plausible even though the README never walks through one.

The next step is model download. The README recommends the 3.0 checkpoint for better performance and shows both ModelScope and Hugging Face calls. For Hugging Face, the lines are:

from huggingface_hub import snapshot_download snapshot_download('FunAudioLLM/Fun-CosyVoice3-0.5B-2512', local_dir='pretrained_models/Fun-CosyVoice3-0.5B') snapshot_download('FunAudioLLM/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')

CosyVoice README, model download section, checked 2026-10-19

After that the README says to follow example.py, or to start the web UI with python3 webui.py --port 50000 --model_dir pretrained_models/CosyVoice-300M. The README also mentions an optional ttsfrd resource for better text normalization and says that if you skip it, wetext is used by default. Skip it on a Mac. The wheel it installs is named ttsfrd-0.4.2-cp310-cp310-linux_x86_64.whl, and issue #1011 shows that command failing on an M4 Mac mini.

Two more macOS details from the README: the sox troubleshooting commands are apt-get for Ubuntu and yum for CentOS, with no Homebrew line, and the deployment section builds a Docker image and runs it with --runtime=nvidia. The TensorRT-LLM acceleration path and the Docker examples are written for NVIDIA hardware. None of that blocks CPU inference, but none of it is a Mac install guide.

Apple GPU acceleration is not in the default path

Issue #1011 is the clearest public record of the Mac experience. A user on an M4 Mac mini reports the CPU at 100 percent while the GPU sits idle, then quotes the device line in cosyvoice/cli/model.py: cuda if torch.cuda.is_available(), otherwise cpu. There is no MPS branch. Another commenter says support requires changing the code, and later comments show a patch that checks torch.backends.mps.is_available() before falling back.

Forcing MPS is not automatically a win. One commenter hits NotImplementedError for aten::unfold_backward on the MPS device, with the standard advice to set PYTORCH_ENABLE_MPS_FALLBACK=1 so that unsupported operations fall back to CPU. That operator is a gradient operation, so the thread mixes training and inference reports, and other commenters in it say MPS worked after upgrading to a newer PyTorch. Others report that newer PyTorch and newer macOS are needed, and that results vary by version. In issue #134 (July 2024, before CosyVoice 2 and 3), a project collaborator writes that an earlier MPS pull request was slower than CPU on Mac and that it was maybe not mature yet. These are user reports and maintainer comments, not a supported configuration.

The practical sequence is therefore: get CPU inference working first with the 0.5B checkpoint, confirm that a short line synthesizes and saves, then try an MPS patch on a copy of the repository and compare wall-clock time against CPU. Do not start by patching a fresh clone before you know the baseline works. And do not install torch nightly over the pinned 2.3.1 in the project environment without a separate environment to fall back to.

Should you run it yourself?

The honest summary: CosyVoice is worth the setup if you specifically need its multilingual and dialect behavior, or you want to study or modify the model. It is the wrong tool if you want a finished desktop editor, if you need Windows, or if you need a supported Apple GPU path today. The local studio versus DIY comparison covers that trade in more detail.

What can go wrong

• The install assumes Linux. The sox fixes are apt-get and yum, and the optional ttsfrd wheel is linux_x86_64 only. On macOS, skip ttsfrd and accept the wetext fallback.
• The default device selection is CUDA or CPU. On a Mac you get CPU until you patch the device lines, which is why Activity Monitor can show a busy CPU and an idle GPU.
• MPS can fail on specific operators. The unfold_backward error in issue #1011 is one example, and the suggested fallback variable makes those operations run on CPU, while other commenters there report MPS working on newer PyTorch.
• Speed is not guaranteed by the patch. A collaborator in issue #134 reports that an earlier MPS path was slower than CPU, so measure before you commit.
• Version drift breaks installs. The requirements pin torch 2.3.1, while MPS comments ask for much newer PyTorch. Keep separate environments.
• The GPU deployment paths are NVIDIA-oriented. vLLM, TensorRT-LLM, and the Docker examples are not macOS acceleration.
• The papers list model limits. CosyVoice 2 supports a limited language set, neither paper controls timbre through text instructions, and singing is described as a weak case.
• Checkpoints take disk space. You may end up with 300M, 0.5B, and other weights side by side, plus the repository and its submodules. Plan the folder before downloading.
• The README's academic-purposes disclaimer sits next to an Apache-2.0 license. Read both, plus the model card, before commercial use.

License, consent, and commercial use

The repository LICENSE is Apache License 2.0, and the CosyVoice2-0.5B and Fun-CosyVoice3-0.5B-2512 model cards both list apache-2.0. Apache-2.0 permits commercial use and modification, with notice and attribution conditions. It does not settle the rights in your reference audio, and it does not override the README disclaimer. The open source model license checklist walks through the questions to ask before shipping.

Voice cloning adds a second layer. Use recordings you made, recordings you licensed, or recordings you have written permission to imitate. Do not clone a public figure or a private person to imply endorsement, and do not use a cloned voice for fraud or harassment. If a client supplies the reference, get the permission in writing and keep it with the project files.

Frequently asked questions