How AI Voice Generators Work

Published September 2, 2026 · This is not a general explanation of speech synthesis. It is what ten named vendors publish about their own systems, quoted from their documentation and read on September 2, 2026. Where a vendor documents nothing, we say so rather than filling the gap. Resemble AI is included because its documentation and open models remain published, but the company no longer sells voice generation.

The short answer

On this page

Reading the machine from its controls

Most explanations of this technology are written from the outside, and they are all roughly the same explanation. This one is written from the documentation, which turns out to be more useful, because a vendor that will not describe its model will still document every knob attached to it — and the knobs tell you where the stages are.

Read across all ten providers and the same pipeline appears, because they all expose settings for the same four things before the audio arrives:

1. Normalization — your text is rewritten into speakable form. “55” becomes “fifty five”.
2. Pronunciation — a layer where you override how particular words are said, by dictionary, by phonetic alphabet, or in one case by prose instruction.
3. Prosody — pace, pauses, emphasis, and sometimes emotion, controlled by markup or by parameters.
4. Voice — a stored representation selected per request.

Then audio comes out in a format and sample rate you choose. What happens between the fourth stage and the audio is the part the documentation mostly goes quiet about, and that section comes last.

Stage one: the normalizer rewrites your text

Before any model sees your text, something rewrites it. This is the least-known stage and the most likely to surprise you, because it is usually on by default and it changes what gets said.

Cartesia gives the clearest published description: “Text normalization converts written forms into spoken forms: 7:00 PM is spoken as ‘seven PM’, and (415) 555-1212 is read as a phone number rather than a twelve-digit string. It runs automatically on every TTS request before the model.”

It is also the only vendor here that lets you pin the normalizer to a locale independently of the voice: its normalization field takes auto, off, or a locale code such as en-IN to “read dates and numbers the Indian English way”. If you generate English for an Indian audience, that is a control nobody else documents.

The Speechify entry is a genuine defect worth acting on. The same text_normalization parameter, carrying a word-for-word identical description, is documented with opposite defaults on its non-streaming and streaming endpoints. A caller moving from /v1/audio/speech to /v1/audio/stream would silently lose normalization — numbers that were being spoken as words would start being spoken as digits, with no error and no warning.

ElevenLabs contradicts itself more mildly. Its best-practices guide says “Normalization is enabled by default for all TTS models”, while its API reference documents the default as auto, under which “the system will automatically decide whether to apply text normalization”. Always-on and the-system-decides are different behaviours. It also warns that the separate language normalization parameter “can heavily increase” latency, which is a real trade-off stated plainly.

Two vendors tie normalization to cost or speed. MiniMax states its normalizer “Improves performance in digit-reading scenarios at the cost of slightly higher latency” and defaults it off. Speechify notes the same latency effect. So on those platforms, correct number reading is something you opt into and pay for in time.

Amazon’s normalizer is language-aware in a way worth knowing. It interprets formatting without any tag — “if your text includes ‘202-555-1212,’ Amazon Polly interprets it as a 10-digit telephone number” — and the rules differ by locale: “in UK English, sequences of the same digit are grouped together, as in ‘double five’ or ‘triple four’”. The same input produces different speech depending on the voice you selected.

Stage two: the pronunciation layer

Every vendor here gives you some way to fix a word the model says wrongly, and the mechanisms fall into three families.

Dictionary files. ElevenLabs uses W3C pronunciation lexicon files: “Pronunciation dictionaries use XML-based .pls files”, supporting “both IPA and CMU alphabets”. It carries a model restriction that will catch people out, and the vendor states it three different ways — one page says phoneme tags “only work with eleven_flash_v2 and eleven_v3 models”, another restricts them to eleven_flash_v2 alone, and two help-centre pages name Flash v2 and Turbo v2. Whichever reading is right, a dictionary can silently stop applying when you switch model. Cartesia takes a simpler approach and says so: “A dictionary is a simple search and replace, which directs the model to use another string in lieu of the text from the transcript”, accepting either IPA or “sounds-like” guidance, referenced per request by pronunciation_dict_id.

Markup. Speechify offers only the SSML sub tag, which “replaces pronunciation for contained text”. Resemble documents alias the same way. This is the narrowest option: it fixes a word in one request rather than establishing a rule.

Prose instructions. OpenAI has no lexicon and no dictionary. You write pronunciation guidance into the prompt, and its documentation shows exactly that: “# Reference Pronunciations When voicing these words, use the respective pronunciations: - Pronounce ‘SQL’ as ‘sequel.’ - Pronounce ‘PostgreSQL’ as ‘post-gress.’ - Pronounce ‘Kyiv’ as ‘KEE-iv.’” Its own guidance on scale is candid: “Keep to a short list; update as you hear errors.”

That difference is architectural, not cosmetic. A dictionary is a lookup applied deterministically; a prompt instruction is another input to a language model, which may or may not follow it. If you have two hundred product names to get right, those are very different propositions.

ElevenLabs contradicts itself on which models accept phoneme tags: its best-practices guide restricts them to eleven_flash_v2 alone, while two help-centre pages state that Flash v2 and Turbo v2 both support them. Both readings were confirmed verbatim on the same day.

Stage three: prosody, and why SSML is not a standard here

Speech Synthesis Markup Language is a W3C specification, which creates a reasonable expectation that markup written for one vendor works with another. It does not.

Four different situations sit in that table. Azure and Google implement the W3C specification partially and say which parts. Amazon implements it per engine, so the same markup behaves differently depending on a parameter elsewhere in your request. Cartesia and Murf have invented their own syntaxes. OpenAI has replaced markup with prose, and MiniMax documents no prosody control at all.

The ElevenLabs row is the one to watch if you are upgrading. Break tags work, except on the model the vendor has made generally available and positions at the top of its catalog. Moving to the newer model silently drops your pause control, and the documentation directs you to prompting techniques instead.

Stage four: what a voice actually is

A voice is a stored thing you reference per request, and the vendors describe what that thing is with unusual clarity — more clarity, in several cases, than they give their models.

ElevenLabs publishes the best explanation in this market, and it is worth quoting in full because it settles a common misconception: “Voice cloning captures a representation of a voice, not a recording of it. The system learns patterns — formant frequencies, prosodic tendencies, spectral characteristics — from your audio samples and encodes them”. It also distinguishes the two cloning tiers mechanically rather than commercially: “Instant Voice Cloning works through few-shot adaptation: the model uses your audio sample as a conditioning signal at inference time, adjusting its output to match the target voice without any model weight updates.”

That last clause is the real distinction between the tiers described in our guide to how AI voice cloning works. Instant cloning conditions a general model at inference time; professional cloning changes weights. Cartesia describes its professional tier the same way — it “fine-tunes a Cartesia text-to-speech (TTS) model on your audio” — which is why one takes seconds and the other takes hours.

Google stores the voice on your side, not its own. Its instant custom voice “takes the form of a voice cloning key, which is a text-string representation of your voice data”, and consequently: “There is no limit to the number of voice cloning keys that you can create, because voice cloning keys are stored on the client side and provided per request.” That is a genuinely different architecture from every other provider here, and it is why the usual per-account voice limits do not apply.

Azure exposes the embedding as markup. “The user’s voice characteristics are encoded in the speakerProfileId”, and that identifier is passed through an SSML element, <mstts:ttsembedding>. The voice representation is literally a value in your markup.

Cartesia migrates voices forward; Resemble ties them to a version. Cartesia states that “As Cartesia releases newer models, we automatically make your voice available on them, so it keeps working when you upgrade.” Resemble does the opposite by design: “The synthesis API automatically uses the model associated with your voice_uuid”. Both are defensible; they are opposite answers to what happens when the model beneath your voice changes.

Stage five: what comes out

The last stage is the best-documented in the market, because it is the one every integration touches.

Formats and sample rates are published by all ten. ElevenLabs encodes both into a single string, and explains the convention: “Formatted as codec_sample_rate_bitrate. So an mp3 with 22.05kHz sample rate at 32kbs is represented as mp3_22050_32” — with 28 enumerated values. Murf publishes MP3, WAV, FLAC, ALAW, ULAW, PCM and OGG at 8000, 24000, 44100 or 48000 Hz. MiniMax supports six sample rates and four bitrates, noting the bitrate “only applies to audio in mp3 format”.

Two constraints are easy to miss. Cartesia’s container support depends on the transport: “Only the Bytes endpoint supports all container formats; our other endpoints (SSE, WebSockets) only support raw” — so choosing streaming means choosing raw PCM and handling the container yourself. And Speechify warns that its default may move: “the current default is ‘wav’, but there’s no guarantee it will not change in the future. We recommend always passing the specific param you expect.” That is good advice generally, and it is the only vendor here that gives it.

The part most vendors will not describe

Everything above is documented because it is configurable. The model itself is not configurable, and seven of ten vendors say something about it while three say essentially nothing.

Amazon Polly is by a wide margin the most forthcoming, and describes four engines differently. Of the standard engine: it “concatenates phonemes of recorded speech” using “concatenative synthesis”. Of the neural engine: “The Amazon Polly NTTS engine doesn’t use standard concatenative synthesis to produce speech. It has two parts” — “A neural network — that converts a sequence of phonemes (the most basic units of language) into a sequence of spectrograms”, then “A vocoder — that converts spectrograms into a nearly continuous audio signal.” Of the generative engine, the most specific architecture statement we found anywhere: “It deploys a billion-parameter transformer that converts raw text into speech codes, followed by a convolution-based decoder that converts these speech codes into waveforms in an incremental, streamable manner.” A safety note elsewhere adds that it renders “speech token by token”.

Google describes a distinction between voice families, including how the older ones work: “One common speech technology, parametric text-to-speech, typically generates audio data by passing outputs through signal processing algorithms known as vocoders”, against WaveNet models “trained using raw audio samples of actual humans speaking”. It also connects two products: “Neural2 voices are based on the same technology used to create a Custom Voice.”

Resemble publishes parameter counts and an optimization for its open models — a “350M parameter architecture” for Turbo and “an even smaller 110M parameter package” for Nano, and: “We have also distilled the speech-token-to-mel decoder, previously a bottleneck, reducing generation from 10 steps to just one.”

Cartesia names an architecture, in marketing rather than documentation: “Our models are designed for live interactions, built on State Space Models (SSMs), a new primitive for low latency, long-context reasoning, and greater efficiency at scale.” OpenAI names a lineage: “GPT-4o Mini TTS is a text-to-speech model built on GPT-4o Mini, a fast and powerful language model.”

Three publish nothing. We found no architecture statement from ElevenLabs, Speechify or MiniMax beyond promotional adjectives. That is not a criticism — no vendor owes you its model design — but it is worth noticing that the market’s most-discussed name explains its models least, while the cloud incumbent explains them most. And ElevenLabs, which says nothing about its architecture, gives the clearest published account of how a cloned voice is represented. The silence is not uniform, and it does not track with quality of documentation generally.

Will the same text give the same audio?

A practical question with almost no published answer. Across all ten providers we found no clear statement that identical input yields identical output, and no seed parameter documented for text-to-speech.

What we did find is the opposite signal. Resemble exposes temperature, documented as controlling “generation randomness (0.1–5.0)” with a default of 0.8 — a non-zero default randomness. Amazon describes its generative engine as rendering “speech token by token” and refers to “the unlikely event of model hallucination”, for which “an imposed emergency stop mechanism is in place”.

The practical consequence: if your workflow assumes regenerating a line produces the same audio, verify that assumption on your provider before relying on it. Nothing in the documentation we read guarantees it.

Using this when something goes wrong

The value of knowing the stages is that a defect belongs to one of them, and each has a different fix.

1. A number, date or phone number is read wrongly — that is the normalizer. Check whether it is on, and on which endpoint. If the same text behaves differently on your streaming and non-streaming calls, check the parameter default on each; at least one vendor documents them differently.
2. A name or technical term is mispronounced — that is the pronunciation layer. Use a dictionary where one exists; check your model still supports phoneme tags after an upgrade.
3. Pauses or pacing are wrong — that is prosody. Confirm your markup is supported by the specific engine and model you are calling, not just by the vendor.
4. It changed after a model upgrade — that is the voice representation. Ask whether your voice migrated to the new model or stayed pinned to the old one. Vendors answer that question in opposite directions.
5. Streaming sounds different from batch — check container and encoding first, and normalization defaults second.
6. It varies between identical runs — look for a randomness parameter, and do not assume determinism.

Frequently asked questions

Sources and what we could not verify

Every quotation on this page was read from the vendor’s own documentation on September 2, 2026. Vendor promotional adjectives are marked as such where quoted and are not assessments by this site. Per-provider detail is in the ten reviews linked throughout; integration limits are in our API comparison; cloning requirements are in our guide to how AI voice cloning works. The method is on the methodology page.

What we could not verify

Honesty about gaps beats a page that looks complete. Genuinely unresolved as of September 2, 2026:

• How any of these systems actually works internally. This page reports what vendors publish about their own pipelines. Where a vendor is silent we have left the silence rather than explaining on its behalf, and nothing here is drawn from general knowledge of speech synthesis.
• The architecture of three providers. We found no architecture statement from ElevenLabs, Speechify or MiniMax beyond promotional adjectives.
• Whether any provider is deterministic. No vendor states it, and none documents a seed for text to speech.
• Speechify’s true normalization default. Two of its own endpoints document the same parameter with opposite defaults.
• Which ElevenLabs models accept phoneme tags. Its best-practices guide and two help-centre pages disagree, and all were confirmed live the same day.
• Whether Cartesia’s normalization behaviour belongs to Sonic 3.5 or 3.6. Two of its pages attribute it to different model generations.
• How any of this sounds. No listening test has been run. A documented control is not evidence that it works well.

Change log

September 2, 2026 — First publication. All quotations read from official vendor documentation on September 2, 2026 and dated accordingly.

Published September 2, 2026 · This page shows no audio and reports no listening results, because our benchmark has not run. Vendor documentation changes without notice and several of the parameters quoted here are model-specific; corrections are recorded with their date on the corrections page. Independence note: this page contains no affiliate links, and no vendor paid for placement or influenced the order — see how we make money and our editorial policy.