How to Measure Streaming TTS Latency on a Mac

Direct answer: measure streaming TTS with at least five separate numbers: time from submit to first usable audio, time to continuous playback, underrun count and duration, total generation time, and real-time factor. Add source completion, peak memory, cancellation time, and join quality. First buffer latency alone can reward a system that starts quickly and then stalls. Run one cold request and at least 30 warm requests on fixed short, medium, and long scripts. Capture the model, runtime, precision, voice, sample rate, chunk policy, audio device, and Mac power state. A streaming system passes only when the listener receives correct, continuous audio.

Streaming metrics

Instrument one shared timeline

Use a monotonic clock and record submit, normalized text ready, model ready, first generated samples, first queued audio, playback start, each underrun, generation complete, playback complete, cancel request, and resource release. Do not mix wall-clock timestamps from separate processes without synchronization. Store event names and durations, not private text. A timeline exposes whether delay comes from the model, decoder, buffer policy, or audio output.

Use scripts that stress different behavior

Include a one-sentence response, a 250-word narration, and a multi-section long-form script. Add names, numbers, punctuation, and a critical final phrase. Short text tests fixed startup overhead. Medium text reveals buffer stability. Long text reveals drift, memory growth, and chunk joins. Use the blind completion test so fast output cannot pass after omitting words.

Define a usable buffer

The first model samples may be too short to play without a gap. Choose a minimum buffer policy based on measured generation variability and output-device behavior. Report both first generated samples and first continuous playback. Count every underrun and its duration. Test after a cold launch, after warmup, and while another audio or GPU application runs. Record headphones or output routing if it changes initialization.

Publish distributions and failure rates

For each script, report p50 and p95 first usable audio, total time, real-time factor, underruns, completion pass rate, peak memory, and cancellation. Keep cold results separate. Averages hide occasional multi-second stalls. Compare systems only with matched scripts, voices, formats, and quality gates. Link raw rows to the Mac benchmark disclosure rather than combining vendor headline numbers.

Production acceptance record

Before acting on this measure streaming TTS latency Mac recommendation, create an acceptance record for the exact feature and release. Name the owner, reviewer, user task, minimum Mac, operating system, application version, runtime, model revision, precision, input fixture, expected artifact, and pass threshold. Attach shared monotonic event timeline, short, medium, and long scripts, thirty warm runs per script. Keep upstream documentation in one section and measurements from your own machine in another. Record every unsupported path, timeout, crash, incorrect output, and manual repair. A result counts as a pass only when the artifact is usable for the stated job. Fast but incomplete output, an unrecoverable cancellation, or a file that cannot be reopened remains a failure.

Run the record once on a clean launch, once after warmup, once with realistic companion applications open, and once after deliberately interrupting the workflow. Exercise the operating steps that matter most here: Use a monotonic clock. Mark every generation and playback stage. Define first usable audio. Save raw timings and checks instead of copying only a winning screenshot. Include rejected options and the reason each lost, so a later framework or model update can reopen the decision without starting from memory. Assign an update trigger such as a new runtime release, model revision, minimum operating system, material support issue, or change in licensing. Review the record before changing defaults, removing a fallback, or publishing a broad performance claim.

Evidence to collect before publication

Add timestamped instrumentation to one local streaming engine and publish raw CSV for 30 warm runs per script length plus a cold run. Record audio-buffer events, compute release after cancellation, and an aligned transcript for completion. Include one run under competing GPU or audio load and identify any underrun without hiding it from the summary.

• Shared monotonic event timeline
• Short, medium, and long scripts
• Thirty warm runs per script
• Underrun and completion results
• Cancellation and resource-release timing

Reader worksheet

Write the user's latency expectation, first usable audio definition, minimum buffer, tolerated underruns, completion gate, cancellation target, output format, and tested hardware. Add upstream claims in a separate column from Murmur measurements. Use the TTSKit guide as one concrete Swift test path.

Practical checklist

1. Use a monotonic clock.
2. Mark every generation and playback stage.
3. Define first usable audio.
4. Count underruns and missing words.
5. Separate cold and warm results.
6. Report p50, p95, failures, and cancellation.

Frequently asked questions