How to evaluate text-to-speech voices before you ship them

Picking a voice for a product usually goes like this: open the provider's demo page, type "Hello, how are you today?", listen to six voices, pick the one that sounds nicest. Then the voice goes into a support agent that has to read order numbers, or a narrator that has to get through Urdu place names, and it falls apart. Or it sounds right but takes two seconds before anything plays, which in a conversation feels like forever.

I built Signal Tank to make that choice with evidence. You write or pick a real script, choose a voice and settings, generate a take, and every take is saved next to its numbers so you can compare them side by side. The code is small and public, and the patterns in it apply to any TTS provider.

Signal Tank on GitHub: A latency-aware text-to-speech lab: server proxy, preset voices, safe cloning and Web Audio

The lab: script, voice, delivery, review. Every take stays in the session for comparison.

Test with scripts that look like production

Every take is a row.

The lab ships five sample scripts: narration, a support agent reading structured details, Urdu, Spanish and Arabic. They exist because the failures I cared about only show up with real content: numbers and codes read wrong, a switch between languages, a long sentence where the voice loses energy halfway. A greeting tests none of that.

Every take stores the script, the voice, the settings (format, latency mode, temperature, top-p, speed, volume) and two timings. Comparing voices then becomes comparing rows, not memories.

Two numbers, not one

Two numbers, not one.

"Latency" for TTS hides two different numbers:

• Time to first byte is how long until audio starts arriving. With streaming playback, this is how fast the product feels.
• Time to complete is how long until the whole file has arrived. If your player needs the complete file, this is what the user actually waits for.

A voice can win one and lose the other, and the latency mode you choose shifts the balance. The browser can measure both by reading the response as a stream instead of calling response.blob():

public/app.js

const started = performance.now() const response = await fetch('/api/tts', { method: 'POST', body: data }) const reader = response.body.getReader() const chunks = [] let firstByte while (true) { const { value, done } = await reader.read() if (done) break if (firstByte === undefined) firstByte = performance.now() - started chunks.push(value) } const total = performance.now() - started const blob = new Blob(chunks, { type: response.headers.get('content-type') }) 

Measured now, playable later.

performance.now() is monotonic and sub-millisecond, unlike Date.now(). Both numbers include the whole round trip through your own server, which is what a user experiences, so measure from the browser rather than trusting the provider's dashboard.

One limit: the lab currently waits for the complete file before playing it. That is fine for comparing voices, but it means the "first byte" number is a measurement, not something you can hear yet. Incremental playback through Media Source Extensions or an AudioWorklet is the next step.

Keep the key on the server

The browser never talks to the TTS provider. It posts a form to /api/tts, a Vercel function that holds the key, validates everything, and streams the audio back:

api/tts.js

let bytes = 0 const counted = data.pipeThrough(new TransformStream({ transform(chunk, controller) { bytes += chunk.byteLength controller.enqueue(chunk) }, flush() { log('signal.tts.complete', { requestId, bytes, durationMs: Math.round(performance.now() - started) }) }, })) return new Response(counted, { status: 200, headers: { 'Cache-Control': 'no-store', 'Content-Type': contentType, 'X-Request-ID': requestId }, }) 

The TransformStream passes chunks straight through, so streaming is not delayed, while it counts bytes and logs the total when the stream ends. The server logs when headers arrived and when the body finished, which gives you the provider's share of the latency separately from the network to the user. The request ID goes back in a header so a slow take in the UI can be matched to its log line. Logs carry character counts and settings only, never the script, and a test checks that.

Do not let the client pick arbitrary voices

An early version sent the provider's voice reference ID from the browser. That meant anyone could call my endpoint, on my credits, with any voice in the provider's library, including voices cloned from other people. Now the server owns a frozen allowlist and the client sends only a preset name:

lib/fish.js

export const VOICE_PRESETS = Object.freeze({ balanced: { label: 'Balanced' }, 'female-american': { label: 'Female · American', referenceId: '…' }, 'male-british': { label: 'Male · British', referenceId: '…' }, // … }) if (form.getAll('voicePreset').length !== 1 || !VOICE_PRESETS[voicePreset]) { throw inputError('Choose one available voice profile.') } if (referenceId) throw inputError('Manual voice references are not supported.') 

The client picks a name, never an ID.

getAll(...).length !== 1 is a small check worth stealing. Multipart forms allow the same field more than once, and different layers may read the first or the last value. Requiring exactly one closes that gap. Every numeric setting is range-checked too: temperature and top-p 0 to 1, speed 0.5 to 2, volume -20 to 20.

Map provider errors to messages you control

Upstream errors are translated into a short, fixed set: a rejected key, no credits (402), rejected settings (422), busy (429), and everything else as a generic 502. The user gets a useful message, and nothing from the provider's internals leaks into the UI. A missing key returns a clear 503 telling you which environment variable to set.

Recording reference audio for a cloned voice

What the recorder watches for.

Cloning needs a clean sample of the speaker, and most bad clones come from bad samples. The recorder does three things to help:

1. Pick a format the browser can actually record

Safari and Chrome disagree, so the recorder tries audio/webm;codecs=opus, then audio/mp4, then audio/ogg;codecs=opus, and uses the first that MediaRecorder.isTypeSupported accepts.

1. Show a live level and warn about silence

A Web Audio AnalyserNode (fftSize 256) computes an RMS level every frame. If the smoothed level stays under 0.035 for two seconds, the UI says "No sound detected, check your microphone or speak closer."

1. Keep the sample in a useful length

At least 10 seconds, stopped automatically at 30, with 15 to 20 recommended. Files are capped at 4 MB to stay under the Vercel request body limit.

The level meter is a few lines:

analyser.getByteTimeDomainData(samples) let energy = 0 for (const s of samples) { const centered = (s - 128) / 128 energy += centered * centered } const level = Math.min(1, Math.max(0, (Math.sqrt(energy / samples.length) - 0.012) * 8)) smoothed += (level - smoothed) * 0.24 

Cloning also requires an explicit consent checkbox and the exact transcript of what was read. The transcript improves the clone, and the consent step is the minimum for working with someone's voice.

A checklist before you pick a voice

• Use your real scripts, including the awkward ones: numbers, codes, names, a second language.
• Measure time to first byte and time to complete from the browser, and record which latency mode produced them.
• Run each voice more than once. A single take tells you very little about variance.
• Listen on the device your users have. Laptop speakers hide a lot.
• Keep keys and voice IDs on the server, and allowlist what the client may choose.
• Log sizes and timings, never the text people typed.

git clone https://github.com/mahmadmanzoor/signal-tank cd signal-tank && npm install cp .env.example .env # add your key npm start # http://127.0.0.1:3000/lab 

Without a key the lab still runs and the generate button returns a clear message, so you can try the UI first.

Frequently asked questions

How do you measure text-to-speech latency?

Measure two numbers from the browser: time to first byte and time to complete. Read the response as a stream instead of calling response.blob(), and time it with performance.now(), which is monotonic and sub-millisecond. Measuring from the browser includes the round trip through your own server, which is what a user actually experiences.

What is the difference between time to first byte and time to complete in TTS?

Time to first byte is how long until audio starts arriving, which sets how fast a streaming product feels. Time to complete is how long until the whole file has arrived, which is what the user waits for if your player needs the full file. A voice can win one and lose the other.

How should I choose a text-to-speech voice for a product?

Test with scripts that look like production, not a greeting. In Signal Tank I use narration, a support agent reading structured details, Urdu, Spanish and Arabic, because numbers, codes and language switches are where voices fail. Save every take with its settings and timings, and run each voice more than once to see variance.

Should the browser call the TTS provider directly?

No. Keep the API key on the server and have the browser post to your own endpoint, which validates input and streams the audio back. Let the client send only a preset name, and map it to a voice ID from a frozen server-side allowlist, so nobody can use your credits with any voice in the provider's library.

How do I record good reference audio for a cloned voice in the browser?

Pick a format the browser can record by trying audio/webm;codecs=opus, then audio/mp4, then audio/ogg;codecs=opus with MediaRecorder.isTypeSupported. Show a live level meter and warn if the smoothed level stays under 0.035 for two seconds. Require a consent checkbox and the exact transcript of what was read.

Originally published on my site. More posts and projects at mahmadmanzoor.com.