---
title: "AuK Model Guide: Open-Source TTS, Speech Editing & AuK-Flash | Murmur"
description: "Learn how AuK works and what it can do. Covers AuK-Flash, zero-shot TTS, Instruct TTS, speech and lyric editing, emotion, timbre, enhancement, separation, setup, and all released task examples."
canonical: "https://www.murmurtts.com/blog/auk-open-source-speech-model-guide"
---
[Murmur](https://www.murmurtts.com/)/[Blog](https://www.murmurtts.com/blog)/Model Guide

Model Guide

# AuK: A Practical Guide to the Open-Source Speech Generation and Editing Model

A hands-on guide to AuK and AuK-Flash: architecture, setup, text-to-speech, speech and lyric editing, enhancement, separation, paralinguistic controls, acoustic edits, and every task example released by the project.

![](https://www.murmurtts.com/murmur-icon.webp)Murmur·September 13, 2026·13 min read

On this page

AuK and AuK-Flash at a glanceHow the model worksEvery task AuK currently exposes1. Instruct TTS: describe the voice you want2. Zero-shot TTS: clone a voice from a reference clip3. Speech content editing: replace, insert, or delete words4. Lyric editing: change sung words without rebuilding the song5. Pitch editing6. Speed editing7. Volume editing8. Emotion editing9. Timbre editing10. De-accent editing11. Nonverbal editing12. Whisper conversion13. Speech enhancement14. Speech separation15. Music separation16. Target speaker extractionThe common interface is the real design ideaHow to run AuKGradio, ComfyUI, CLI, and Python are all first-class pathsWhat the benchmark claims actually sayWhere AuK is especially interestingWhat to keep in mind before building around itLicense and availabilityWhy AuK matters for speech software

[Back to the journal](https://www.murmurtts.com/blog)

**What AuK is**

AuK is a 1.5B-parameter open-source foundation model for speech generation and editing. Instead of shipping separate models for TTS, denoising, speaker separation, emotion changes, pitch edits, and content edits, it exposes those jobs through the same natural-language instruction interface. Tencent Hunyuan open-sourced the code and weights on September 9, 2026, one day after the technical report appeared on arXiv.

The interesting part of AuK is not simply that it can synthesize speech. It treats speech creation and speech editing as variations of the same problem: give the model an instruction, optionally give it audio context, and ask it to generate the desired audio. That makes the project closer to an instruction-following audio model than a conventional text-to-speech engine.

The authors report roughly **3.03 billion instruction-audio instances** and **1.95 million hours of effective supervision** spanning five families: speech generation, content editing, enhancement and separation, paralinguistic editing, and acoustic editing. The project releases two variants: the full AuK model for quality-focused generation and **AuK-Flash**, a distilled version built for fast four-step inference.

## AuK and AuK-Flash at a glance

| Area | AuK | AuK-Flash |
| --- | --- | --- |
| Model role | Full 1.5B foundation model | Distilled fast-inference variant |
| Inference steps | Configurable NFE | Fixed at 4 steps |
| Classifier-free guidance | Configurable | CFG = 0 at inference |
| Primary goal | Highest generation/editing quality | Near-teacher quality with lower latency |
| Reported speed | Baseline | 4.5× wall-clock speedup under matched conditions |
| Task coverage | Generation + editing + restoration + separation | Same general instruction-driven interface |

The paper says AuK-Flash is produced in two stages: trajectory-level consistency initialization first creates a few-step student, then a task-routed Decoupled DMD objective further distills it. Separation also receives clean-prediction regression. The result is a four-step model that does not need classifier-free guidance during inference.

## How the model works

AuK combines three major pieces. A multimodal large language model handles semantic conditioning from the instruction and optional reference audio. A VAE trained jointly on speech, general audio, and music represents acoustic information at 50 Hz. A hybrid rectified-flow Transformer then generates the target audio using dual-stream MMDiT blocks followed by unified single-stream DiT blocks.

- Semantic conditioning: a multimodal language model interprets what you want the audio to become.
- Acoustic conditioning: a 50 Hz audio VAE carries information about speech and broader audio structure.
- Generation backbone: a hybrid rectified-flow Transformer first processes semantic and acoustic streams separately, then fuses them for final generation.
- Training sequence: generation-only warm-up comes first, followed by joint generation-and-editing pre-training.
- Post-training: human-feedback preference optimization is used for open-ended editing, while reward-based reinforcement learning is applied to speech generation.

That shared architecture is why AuK can accept instructions that would normally belong to very different tools. A command can ask it to speak new text, replace a phrase in an existing recording, remove reverb, keep a particular speaker, make a voice happier, raise pitch by semitones, or turn normal speech into a whisper.

## Every task AuK currently exposes

The official repository groups AuK into five task families and lists sixteen concrete tasks. The table below mirrors that released task set so you can see the full surface area before we walk through each example.

**Listen as you read**

The audio players in the examples below use the official AuK project demo WAV files. For editing tasks, play the source first and then the AuK result to hear what changed.

| Family | Task | What the instruction asks AuK to do |
| --- | --- | --- |
| Speech generation | Zero-shot TTS | Speak new text using the voice in reference audio |
| Speech generation | Instruct TTS | Create speech from a free-form voice description without reference audio |
| Content editing | Speech content editing | Insert, delete, or replace spoken words |
| Content editing | Lyric editing | Insert, delete, or replace sung lyrics while preserving melody and voice |
| Acoustic editing | Pitch editing | Raise or lower pitch by semitones |
| Acoustic editing | Speed editing | Change speaking rate and scale output length with the requested factor |
| Acoustic editing | Volume editing | Raise or lower loudness by a requested dB amount |
| Paralinguistic editing | Emotion | Change emotional delivery while keeping content and voice |
| Paralinguistic editing | Timbre | Change vocal timbre from a description while preserving the words |
| Paralinguistic editing | De-accent | Reduce a regional accent while preserving speaker and content |
| Paralinguistic editing | Nonverbal editing | Add or remove breaths, laughs, coughs, and other nonverbal sounds |
| Paralinguistic editing | Whisper conversion | Convert normal speech to whisper or whisper back to normal speech |
| Enhancement & separation | Speech enhancement | Denoise, dereverberate, or restore cleaner speech |
| Enhancement & separation | Speech separation | Keep one speaker based on speaking order and remove others |
| Enhancement & separation | Music separation | Extract singing voice or retain human vocals from a mix |
| Enhancement & separation | Target speaker extraction | Keep the speaker identified by what that person says |

## 1. Instruct TTS: describe the voice you want

Instruct TTS generates speech from a text description of the desired voice. You do not need a reference recording. The model can condition on details such as apparent age, vocal tone, pace, energy, emotional delivery, clarity, and speaking style, then synthesize the requested sentence.

**Example instruction**

Generate the following line with a calm, warm female voice, clear articulation, a moderate pace, and gentle energy: “Hello, welcome to AuK.”

The official ComfyUI workflow ships with essentially this kind of text-only example. The repository also shows a richer Chinese prompt describing a woman in her twenties speaking gently and affectionately to a partner who has just come home, then asks the model to say a specific welcome-home line. That illustrates the point of Instruct TTS: the voice description can be much more specific than a simple preset such as *warm* or *female*.

The official English Instruct TTS demo asks for a theatrically charged Shakespearean villain voice: a rich baritone, deliberate pacing, crisp staged diction, and a tone that shifts from false sweetness to menace. This is the generated result:

Official AuK demo · Instruct TTS · English output0:00

## 2. Zero-shot TTS: clone a voice from a reference clip

Zero-shot TTS adds reference audio. Instead of describing a voice from scratch, you ask AuK to say new text using the reference speaker's voice. The project page demonstrates this with a short speaker reference and a new Chinese sentence; the README shows the same pattern in English.

**Example instruction**

Say the following with the same voice: “Ladies and gentlemen, it's an honor to have the opportunity to address such a distinguished audience.”

The reference clip supplies speaker identity and acoustic cues, while the instruction supplies the target content. In the CLI, the authors recommend either giving an explicit generation duration or providing text/transcript hints so AuK can estimate the target length.

In the official English demo, AuK receives a reference speaker and generates a new line: *“If not even light can escape the event horizon, how do you think you’ll escape me?”* Compare the reference with the synthesized result:

Official AuK demo · Zero-shot TTS · Reference voice0:00Official AuK demo · Zero-shot TTS · Generated speech0:00

## 3. Speech content editing: replace, insert, or delete words

This is one of AuK's most useful editing modes. Give it an existing recording and describe exactly how the spoken content should change. The goal is to modify the words while keeping the surrounding speaker identity, prosody, and acoustic context consistent.

**Official CLI example**

Replace “but accepting what we cannot have” with “and living well with dreams unmet”.

The task supports replacement, insertion, and deletion. In practical production work, this is the kind of operation you would use to repair a narration line without asking a speaker to re-record an entire paragraph.

Official AuK demo · Speech edit · Original recording0:00Official AuK demo · Speech edit · Replaced phrase output0:00

## 4. Lyric editing: change sung words without rebuilding the song

AuK extends content editing to vocals. Lyric editing can replace, insert, or remove words in a singing recording while attempting to preserve the original melody and vocal identity. That is a much harder constraint than ordinary speech replacement because timing, pitch contour, musical phrasing, and the singer's tone all need to stay coherent.

**Example instruction pattern**

Official demo: replace “rear view” with “like you” in the lyrics while preserving the singer, melody, timing, and surrounding music.

Official AuK demo · Lyric edit · Original song excerpt0:00Official AuK demo · Lyric edit · Edited lyrics0:00

## 5. Pitch editing

Pitch editing changes the perceived pitch of the speech by a specified number of semitones. Because this lives in the same instruction interface, the request can be expressed directly instead of selecting a dedicated pitch-shifter tool.

**Example instruction pattern**

Raise the pitch of this speech by 3 semitones while keeping the words and timing unchanged.

Official AuK demo · Pitch · Source0:00Official AuK demo · Pitch · +3 semitones0:00

## 6. Speed editing

Speed editing changes speaking rate. The repository notes that output duration scales with the requested speed factor, so this is a structural timing change rather than an instruction to merely sound more energetic.

**Example instruction pattern**

Make this speech 1.25× faster while preserving the speaker and spoken content.

Official AuK demo · Speed · 1.0× source0:00Official AuK demo · Speed · 1.25× output0:00

## 7. Volume editing

Volume editing asks the model to increase or decrease level by a requested decibel amount. It is the simplest of the acoustic edits conceptually, but its inclusion matters because AuK tries to make even signal-level operations available through the same language-driven interface.

**Example instruction pattern**

Increase the volume by 5 dB without changing the speaker, timing, or content.

Official AuK demo · Volume · 0 dB source0:00Official AuK demo · Volume · +5 dB output0:00

## 8. Emotion editing

Emotion editing changes how a line is delivered while keeping its words and speaker identity. This is useful when the content is already correct but the performance needs a different emotional reading.

**Example instruction pattern**

Say this in a happy tone while preserving the exact words and speaker identity.

Official AuK demo · Emotion · Calm source0:00Official AuK demo · Emotion · Happy output0:00

## 9. Timbre editing

Timbre editing changes vocal color from a natural-language description. The intended content stays the same, but the model can be asked for a brighter, darker, softer, raspier, or otherwise differently characterized voice quality.

**Example instruction pattern**

Official English demo: keep the words and change the timbre to a clean, warm British-accented voice with slow, sincere, story-like delivery.

Official AuK demo · Timbre · Original voice0:00Official AuK demo · Timbre · Edited voice0:00

## 10. De-accent editing

De-accent editing aims to reduce a regional accent while preserving the speaker's identity and the spoken content. This is a particularly demanding example of disentangling *who* is speaking from *how* pronunciation is realized.

**Example instruction pattern**

Remove the regional accent and produce neutral pronunciation while preserving the same speaker and words.

Official AuK demo · De-accent · Accented input0:00Official AuK demo · De-accent · Output0:00

## 11. Nonverbal editing

Nonverbal editing targets sounds that accompany speech rather than the words themselves. The project explicitly lists breaths, laughs, and coughs as examples. The instruction can ask AuK either to remove an unwanted event or add one where it makes sense.

**Example instruction patterns**

Official English demo: add a breath before “We tested” while leaving the spoken words and speaker identity intact.

Official AuK demo · Nonverbal edit · Original0:00Official AuK demo · Nonverbal edit · Breath inserted0:00

## 12. Whisper conversion

Whisper conversion works in both directions: normal speech can be turned into whisper, and whispered speech can be converted back toward normal voiced speech. The task is designed to preserve speaker identity and linguistic content while changing phonation style.

**Example instruction pattern**

Convert this line to a natural whisper while preserving the speaker and exact words.

Official AuK demo · Whisper · Normal speech input0:00Official AuK demo · Whisper · Whisper output0:00

## 13. Speech enhancement

Speech enhancement covers denoising, dereverberation, and general restoration. The official CLI example is written in Chinese and asks AuK to keep all original speakers, remove noise and reverberation, and output clean speech with the same duration.

**Equivalent example instruction**

Restore this audio to clean speech: keep all original speakers, remove noise and reverberation, and return clean speech of the same length.

The authors position restoration as one area where AuK remains competitive with specialized systems, while the strongest claims in the paper are around zero-shot and instruction-controlled generation plus general instruction-guided editing.

Official AuK demo · Speech enhancement · Noisy input0:00Official AuK demo · Speech enhancement · Clean output0:00

## 14. Speech separation

Speech separation lets an instruction identify which talker to retain by speaking order. Instead of choosing an abstract source index, you can ask for the first, second, or otherwise described speaker and remove the others.

**Example instruction pattern**

Official English demo: keep only the first speaker and remove the other speaker.

Official AuK demo · Speech separation · Mixed speakers0:00Official AuK demo · Speech separation · First speaker only0:00

## 15. Music separation

Music separation extends the model beyond spoken recordings. The released task list describes extracting a singing voice from a mix or keeping human vocals while removing the rest of the accompaniment.

**Example instruction patterns**

Extract the singing voice from this music mix. Or: keep all human voices and remove the instrumental accompaniment.

Official AuK demo · Music separation · Full mix0:00Official AuK demo · Music separation · Extracted vocals0:00

## 16. Target speaker extraction

Target speaker extraction identifies the person to keep from what that person says. This gives the instruction semantic grounding: rather than supplying a separate speaker embedding, you can describe the target using a phrase associated with that speaker in the recording.

**Example instruction pattern**

Official English demo: keep only the speaker who says “get what” and remove the other speaker.

Official AuK demo · Target speaker extraction · Mixed speakers0:00Official AuK demo · Target speaker extraction · Content-matched speaker0:00

## The common interface is the real design idea

AuK's Python API uses a message structure that looks more like a multimodal chat request than a traditional audio DSP function. A user message contains text plus optional audio. The same `generate` call is used whether the instruction is TTS, content editing, enhancement, or another supported transformation.

That design lowers the conceptual cost of adding a new operation to a product. Instead of wiring a separate model and parameter panel for every task, an application can expose a common instruction surface and let the underlying model route the request. The tradeoff is that language-driven control can be less deterministic than a fixed DSP operator, so production tools still need careful previews, constraints, and undo/history around destructive edits.

## How to run AuK

The official repository requires Python 3.10 and pins the tested PyTorch family to the 2.7 ABI line. The documented setup clones [Tencent-Hunyuan/AuK](https://github.com/Tencent-Hunyuan/AuK), creates an isolated environment with either `uv` or Conda, and installs the package in editable mode. Optional extras add Gradio, ComfyUI integration, prompt enhancement, ASR dependencies, or fine-tuning support.

1. Clone the Tencent-Hunyuan/AuK repository.
2. Create and activate a Python 3.10 environment with uv or Conda.
3. Install the core package for CLI/Python inference, or add the Gradio/ComfyUI/train extras you need.
4. Download AuK or AuK-Flash weights plus the required Qwen2.5-Omni-3B component into the documented checkpoint layout.
5. Run auk-infer for command-line inference, auk-gradio for the web interface, the supplied ComfyUI workflow, or call AukInfer from Python.

**Base vs Flash**

Use the base checkpoint when quality and configurable sampling matter most. Use AuK-Flash when latency matters more: the repository fixes it to four inference steps with CFG disabled.

## Gradio, ComfyUI, CLI, and Python are all first-class paths

The repository includes an interactive Gradio app that can expose the base model, Flash, or both. It can place the two variants on separate GPUs or load them on one GPU. There is also an AuK Model Loader plus AuK Generate / Edit node for ComfyUI, and the included workflow starts with a three-second text-only example.

For code, the central class is `AukInfer`. You build a message containing instruction text and, for editing or reference-conditioned tasks, an audio item. Calling `engine.generate(...)` returns generated audio and a sample rate, which the helper can then save to disk. This same request shape also forms the basis of AuK's JSONL fine-tuning format.

## What the benchmark claims actually say

The technical report evaluates AuK across generation, editing, enhancement, and separation rather than on one TTS benchmark. For generation it reports Seed-TTS-Eval metrics for intelligibility and speaker similarity plus InstructTTSEval for free-form timbre following. Editing evaluations include MMAE-Speech, SpeechEditBench, and Ming-Freeform-Audio-Edit. Restoration and separation use perceptual metrics including DNSMOS-OVRL and UTMOS across datasets such as DNS Challenge, CHiME-4, and Libri2Mix.

The authors' own summary is appropriately specific: AuK shows leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing, while staying competitive on signal-level restoration tasks. That is a more useful description than calling it universally best at every audio operation.

## Where AuK is especially interesting

- Voice products that want TTS and editing in one model instead of separate generation and repair pipelines.
- Podcast and narration tools where a user may need to replace a phrase, change delivery, remove noise, or isolate a speaker after generation.
- Research on instruction-following audio models, especially systems that mix semantic and low-level acoustic edits.
- ComfyUI workflows that benefit from one node family handling generation and transformations.
- Developers who want inspectable code and weights under a permissive open-source license.

## What to keep in mind before building around it

AuK is extremely new. The code and weights were released on September 9, 2026, so production experience, hardware guidance, third-party benchmarks, and integration patterns are still developing. The repository's documented examples are GPU-oriented, and the current implementation uses Qwen2.5-Omni-3B; its Gradio documentation explicitly says Qwen3-Omni is not currently supported through the corresponding path option.

Also separate model capability from product safety. Voice cloning, target-speaker extraction, and realistic speech editing can be useful creative tools, but an application still needs consent, provenance, access controls, and clear user expectations around how recordings may be transformed.

## License and availability

Tencent releases the publicly available AuK training code, inference code, parameters, and weights under the **MIT License**. The project provides source code on GitHub and model links through Hugging Face and ModelScope. Because license terms can change around related assets or downstream components, check the current repository and any separately downloaded model dependencies before shipping a commercial product.

## Why AuK matters for speech software

For years, voice software has mostly been assembled as a chain of specialists: TTS for generation, a separate editor for content fixes, another model for denoising, another separator for overlapping speakers, and DSP tools for pitch, speed, and gain. AuK asks whether a single instruction-conditioned model can cover enough of that chain to become the common editing layer.

The first release already spans sixteen named tasks, and the interface stays consistent across them. If that approach holds up under broader independent testing, the biggest contribution may be less about any one benchmark and more about collapsing a fragmented speech stack into one programmable model surface.

## Sources

- [AuK project page — model overview, architecture summary, demos, benchmark framing](https://auk-project.github.io/)Accessed 2026-09-13
- [AuK Technical Report — arXiv:2609.08936](https://arxiv.org/abs/2609.08936)Accessed 2026-09-13
- [Tencent-Hunyuan/AuK — source, supported task table, setup, CLI, Python, Gradio, ComfyUI, and fine-tuning examples](https://github.com/Tencent-Hunyuan/AuK)Accessed 2026-09-13
- [AuK MIT License](https://github.com/Tencent-Hunyuan/AuK/blob/main/LICENSE)Accessed 2026-09-13

## Explore speech models on your Mac

Murmur brings multiple speech-generation workflows into a native macOS app with local-first tools for creators.

[Buy Murmur · $49](https://www.murmurtts.com/#pricing)

macOS 15+ · Apple Silicon required · 7-day refund policy

MADE FOR YOUR MAC

## Less reading. *More making.*

[Explore Murmur](https://www.murmurtts.com/#product)