A clean-room, pure-Rust port of Qwen3-TTS-12Hz-0.6B voice cloning. No Python, no PyTorch, no GPU, no server. The same engine that runs faster than real time as a native CLI, compiled to WebAssembly and stitched into this page. Nothing you type or record ever leaves the tab.
Download the model once, then type on the left and listen on the right. The actual engine, compiled to WebAssembly and running in a worker in your browser, synthesizes every sample locally. There is no server behind this page and no mocked demo.
First use downloads the quantized model (≈2.0 GB) into this browser's private storage: verified against pinned SHA-256 digests, resumable, cached until you clear it. After that, the page loads it straight from storage.
Before downloading: the model is 2.0 GB. It lives only in this browser's private storage for this site (nothing is installed on your filesystem), it stays on your device, the download resumes if interrupted, and “Clear model cache” removes it. On a typical 100 Mbps connection this takes about three minutes.
Cloning runs the speaker encoder locally on your recording; the audio is discarded once the 4 KB voice vector exists. Clone only voices you have the right to use.
The share link is stateless: the text, seed, and voice travel inside the URL itself. A preset rides as its name; a cloned voice rides as its full 1,024-float vector plus the name you gave it, in the fragment, so it never touches a server.
Speed honesty, measured: this single-threaded WASM build synthesizes at roughly 0.03–0.05× real time, so a full sentence takes a couple of minutes, with live progress. The native CLI runs the same engine faster than real time; browser threads are the tracked path to closing the gap. Desktop browsers only: the engine needs several GB of memory.
Qwen3-TTS is described as a 12.5 Hz model, which hides its real per-frame cost. The 28-layer transformer runs once per frame; a small 5-layer decoder then runs fifteen sequential times before the codec ever sees a token. This is what one frame actually does.
The microdecoder is small, but rereading its body fifteen times per frame makes it the dominant cost of the whole model. franken_tts treats it as the design center.
A 28-layer transformer predicts each frame's first, semantic-rich codec token. Runs on int8 kernels with exact i32 accumulation, and its output is argmax-verified against the PyTorch oracle through every layer.
A 5-layer residual-code predictor with per-depth embeddings and per-depth 2,048-way heads, run fifteen sequential times inside every frame. Its body accounts for ~1.18 GB of the ≈1.65 GB of Q8 weight traffic each frame touches.
A fully causal decoder turns 16 code groups into 1,920 samples of 24 kHz audio per frame. Streaming output is bit-identical to offline decoding under every packet schedule.
Most TTS systems predict a spectrogram and hand it to a vocoder. Qwen3-TTS works like a language model instead: it writes speech as discrete tokens, sixteen per 80 ms frame, drawn from a residual codec vocabulary. That difference is where the quality comes from, and it is what franken_tts reimplements from scratch in Rust.
A classic TTS stack is several separately trained models handing fragile intermediate formats to each other, and every seam loses information. A codec language model has one seam, and the tokens that cross it are exactly what the codec was trained to decode.
Your text runs through Qwen's BPE tokenizer and is assembled into a prompt with the 1,024-float voice vector and per-token rotary positions. The voice is conditioning, not a filter applied afterwards.
For each 80 ms frame the 28-layer talker predicts one code out of 2,048: the semantic backbone of that slice of speech. What word, what pitch region, what energy.
The microdecoder fills in codes 1 through 15, each depth conditioned on all the previous ones. This is where breathiness, sibilance, and the fine grain of a specific voice live.
A causal convolutional codec upsamples the 16 codes by 8·5·4·3 into 1,920 PCM samples. Causal means it can stream: audio starts before the sentence finishes generating.
A neural codec compresses speech into stacks of sixteen code groups per frame. The first code in each stack carries most of the meaning; every later code encodes only what the ones before it missed. The model then generates these codes autoregressively, the way an LLM generates words.
Because one transformer models content, prosody, pacing, and breaths jointly, the output sounds like a performance rather than a spectrogram stitched to a vocoder. The cost is the nested loop you saw above: fifteen extra decoder passes per frame.
At 12.5 frames per second, a ten-second utterance is only 125 positions of sequence. The whole sentence sits inside the talker's attention window at once, so pitch arcs, emphasis, and pauses are planned across the full line instead of guessed locally.
Sampling runs at temperature 0.9 with top-k 50, seeded. The same text, voice, and seed reproduce the same audio, which is why a share link can stand in for the clip itself.
A speaker encoder distills roughly ten seconds of reference audio into a single 1,024-float x-vector. That vector conditions every frame of generation. There is no fine-tuning and no per-voice training; a new voice is one forward pass of the encoder.
This is the same mechanism the playground uses when you record the enrollment script, and the voice vector is small enough to ride inside a share link.
Drag the slider to rebuild a waveform from its residual codes. Early codes sketch the shape; later ones add detail that the previous codes missed. The shapes here are illustrative; the mechanism is exactly how the sixteen code groups divide their labor.
The arithmetic that makes this hard: at real-time speed the model must stream roughly 20.7 GB of weights per second, first order, even after int8 quantization. Bandwidth like that is why models of this class normally live on GPUs. The port attacks the traffic itself, not just the math.
Anything left of the amber line is slower than the audio it produces. The f32 reference route exists for correctness and stays in the tree as the parity oracle.
The talker and microdecoder run W8A8: symmetric per-channel weights, dynamic per-row activations, and exact i32 accumulation, so quantization halves the bytes without touching the argmax parity gate. The inner loops become integer dot products the CPU accelerates natively.
Six workers stay alive across every layer, every microdecoder pass, and every frame: static partitions, no scheduler, no allocation in the steady state. The partitioning is bit-identical to serial execution at every thread count.
The quantized artifact hydrates int8 weights directly, widens tensors concurrently, and loads the codec and tokenizer in parallel with the talker. Warm model load fell from ~9 s to ~3.7 s.
WebAssembly hands the engine one thread, no memory mapping, no BLAS, no clock in the standard library, and a hard 4 GB ceiling. Getting a 2 GB model to speak under those rules took four separate fixes, each found the hard way. Step through them.
GitHub release downloads send no CORS headers, so a small same-origin proxy forwards 32 MiB Range requests. Chunks stage into the browser's private origin file system, survive a closed tab, verify against pinned SHA-256 digests, and resume where they stopped.
The model bytes transfer into the worker rather than being copied, and the engine hydrates straight from those bytes. On native the artifact is memory mapped; wasm has no files, so the same loader runs from a byte slice instead.
The page already serves the cross-origin isolation headers that unlock SharedArrayBuffer. When the engine's worker team lands in wasm, multi-threaded synthesis needs no infrastructure change. That, plus relaxed SIMD, is the tracked path toward real time in the tab.
Every stage of this engine is checked against captured activations from the pinned PyTorch reference. These are the actual gate results.
The talker's next-token choice matches the oracle exactly, verified layer by layer through the full stack against captured activations.
Whole-utterance codec code sequences match the oracle token for token, and streamed audio is bit-identical to offline decode under all packet schedules.
The int8 route measures 1.4–1.6× real time on a Mac Mini M4 Pro with first audio about 450 ms after synthesis starts, and warm model load at ~3.7 s.
Per-stage activations captured from the pinned PyTorch reference are the ground truth. Tolerances are derived from the oracle's own run-to-run noise floor, never invented.
Verification climbs in order: tokenizer ids exact, then per-layer activations, then argmax decisions, then whole utterances. A kernel lands only after its rung is green.
Every optimization sits behind a kill switch, and the full-precision f32 route stays in the tree. A faster kernel that drifts the output gets reverted, not shipped.
The finished audio's peak frame RMS, this engine vs the pinned PyTorch reference. The bars are drawn to scale; the difference is two parts in a thousand.
Each preset ships inside the binary as a 4 KB x-vector. Pick one below to load it into the playground, or read a thirty-second script into your microphone and make an eighth.
The CLI runs the same engine with native int8 kernels and a persistent six-thread worker team, faster than real time on Apple Silicon.