DEEP DIVE into Directable AI Voices... Too Emotional?

A new text-to-speech model from 11 Labs—its “11 V3 alpha”—is showing unusually tight control over voice performance, including emotion, delivery, and even non-speech audio, while still carrying the instability expected from an alpha. In side-by-side tests, the synthetic voice lands close to the creator’s own timbre, then diverges in small but noticeable ways across generations and voice-clone settings. The biggest leap is how consistently it can maintain a performance style (like whispering in an echoey cave) while also switching tone on command (adding casual delivery without losing the cave acoustics).

The transcript walks through the practical mechanics of that control. Users can insert bracketed “actions” or tags that steer how lines are spoken—ranging from size and exhales to whisper, sarcastic, and sound effects such as applause, gunshot, gulp, and even experimental tags like “fart.” The creator tests whether these tags are truly built-in versus inferred from natural language, and the results suggest the system responds directly to the specified controls rather than merely guessing intent. A whisper-in-a-cave prompt, for example, keeps the echo while blending in a casual tone at the right moment—exactly the kind of multi-parameter direction that would be hard to achieve with more rigid speech models.

Beyond single-line effects, the model’s expressiveness shows up in longer, character-like dialogue. The creator stages a “goblin” scene with aggressive, confident delivery and hears convincing acting through intonation and pacing. Still, the alpha has recurring flaws: it sometimes rushes through segments, cuts off at the end of lines, and produces occasional background weirdness (including music-like artifacts) depending on the generation and voice setting. Attempts to “fix” timing—like adding dashes or copying segments—reduce some issues but can also introduce new oddities, reinforcing that controllability is powerful but not fully polished.

The tests also highlight the model’s broader audio capabilities. The creator compares V3 behavior to earlier multilingual versions and to “expressive instant voice clone” versus “professional voice clone,” noting that longer generations tend to dial in the voice more accurately. There’s also a strong emphasis on the model’s ability to generate full soundscapes—sound effects, dramatic sighs, scurrying, and even outro music—making it feel less like a pure speech engine and more like an open-form audio model.

To demonstrate creative direction, the transcript includes a bracket-guided monologue about becoming a self-employed ice salesman whose ice keeps disappearing, plus a full commercial-style pitch for lemons with tone shifts and structured delivery. The creator reports that the parentheses-based control is “stellar,” even while asking for more dialed-in behavior and fewer cutoffs.

Overall, the core takeaway is that 11 V3 alpha is already capable of producing emotionally directed, character-ready audio with rich sound effects—useful for audiobooks and potentially for interactive audio-to-video workflows—while still needing bug fixes to make the experience reliable enough for production.