Over the past decade, significant breakthroughs in speech synthesis have emerged, driven by the development of neural networks and end-to-end modeling. Last year, Microsoft introduced VALL-E, a neural codec language model capable of synthesizing high-quality personalized speech from just a 3-second recording of an unseen speaker. This model notably outperformed the state-of-the-art zero-shot text-to-speech (TTS) systems at the time.
Building on this progress, in a recent new paper VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers, a Microsoft research team presents VALL-E 2, the latest advancement in neural codec language models. This innovation marks a milestone in zero-shot TTS synthesis by achieving human parity for the first time.
VALL-E 2, an evolution of its predecessor, employs a neural codec language modeling method for speech synthesis and introduces two significant enhancements: repetition-aware sampling and grouped code modeling.
Repetition-aware sampling improves upon the random sampling used in VALL-E by adaptively choosing either random or nucleus sampling for each time step token prediction. This decision is based on the token repetition in the decoding history, enhancing the stability of the decoding process and preventing the infinite loop issue encountered in VALL-E.
Grouped code modeling divides the codec codes into groups, each modeled in a single frame during the autoregressive (AR) modeling process. This approach accelerates inference by reducing sequence length and improves performance by addressing the long context modeling problem.
Notably, VALL-E 2 requires only simple utterance-wise speech-transcription pair data for training, greatly simplifying the data collection and processing. This advancement facilitates potential scalability and streamlines the training process.
Experiments on the LibriSpeech and VCTK datasets demonstrate that VALL-E 2 surpasses previous systems in terms of speech robustness, naturalness, and speaker similarity. It is the first model to achieve human parity on these benchmarks. Furthermore, VALL-E 2 consistently synthesizes high-quality speech, even for sentences that are complex or contain repetitive phrases.
Demos of VALL-E 2 will be posted to https://aka.ms/valle2. The paper VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers is on arXiv.
Author: Hecate He | Editor: Chain Zhang
We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.
Wow, achieving human parity is a huge leap! It’s like the AI can now copy any voice perfectly from just a tiny sample. Makes me wonder how this will change things like audiobooks or helping people who lost their voice. Ginny And Georgia Test
Wow, it can copy any voice from just 3 seconds? That’s like a superpower for making stories come to life! Hope they’re super careful about how it’s used tho. Play X Trench Run Online
Wow, it’s like the AI can now copy a person’s voice from just a tiny sample and make it talk perfectly? That’s kinda mind-blowing and a little scary for the future of audio! Play Pokemon Overlord Online
Wow, it’s like the computer can now copy anyone’s voice perfectly from just a tiny sample? That’s kinda mind-blowing and a little bit scary for making fake stuff. Hope they have good rules for using it. Play Finn’s Fishing Bonanza Online
Wow, it can copy any voice from just a tiny clip? That’s like a superpower for storytime! Makes me think how this could help kids who have trouble speaking to still have their own voice. Play Chudways Online
Wow, it can copy any voice from just a tiny sample? That’s like a superpower for storytime! Makes me think how it could help kids who lost their voice to still sound like themselves. Play A Fly In The Array
Wow, it can copy a voice from just 3 seconds? That’s like a superpower for making computers talk! Sounds like it’s finally getting really, really good at not sounding like a robot. Play Hammy Home
very impressive to see like this and the context and emotional is so good in this now i am gonna confirm this that i will follow your page and see the latest things but need some improvements i am sure you will
I came across vwtoto login while reading online discussions and reviews from other users. What caught my attention was how many people mentioned the smooth access and user-friendly layout. After trying it myself, I can say the experience feels comfortable and modern.
Wow, it’s fascinating to see VALL-E 2 achieving human parity in text-to-speech! How do you think this technology could impact content creation? Also, have you explored how it might enhance interactive experiences like slope run games?
There were a few useful takeaways here that stood out to me. I’ll keep them in mind.
Great job on creating such high-quality content. I learned something new today, and I appreciate the effort that clearly went into writing this article.
Best Dietitian in Chandigarh
Great job on creating such high-quality content. I learned something new today, and I appreciate the effort that clearly went into writing this article.
Luxury meets contemporary fashion with the exclusive Prada sunglasses collection at Kapoor Optical Co., featuring authentic Italian craftsmanship, iconic silhouettes, and premium eyewear designed for a sophisticated everyday look.
Prada sunglasses
The part about VALL-E 2 reaching human parity in zero-shot TTS from just a short recording is wild, especially since VALL-E already set the bar for personalized speech. I’ve been following adjacent generative tools too, and this roundup of Best Free AI Video Generators in 2026 (Real Limits Tested) Best Free AI Video Generators in 2026 (Real Limits Tested) feels like a useful companion read for how fast these models are moving.
Your explanation made the topic much clearer.
The writing is clear and considered. It helps the reader stay engaged from start to finish.
The detail that stands out is repetition-aware sampling—switching between random and nucleus sampling based on token repetition in the decoding history to prevent VALL-E's infinite-loop failures. That adaptive decoding feels as consequential as the headline human-parity result on LibriSpeech and VCTK. Equally interesting is that VALL-E 2 trains on simple utterance-transcription pairs, sidestepping heavier data pipelines. Generative progress is moving quickly across modalities; comparable one-shot convenience for image work shows up in Nano Banana 2.1 at nanobanana-21.si
The claim of “human parity” in zero-shot TTS is striking, but I’d want to see how it was measured before treating it as settled. Human parity can mean different things depending on whether the evaluation relies on subjective listener ratings or objective metrics, and those two often diverge. The jump from VALL-E’s 3-second speaker prompt to this milestone is impressive on paper, but the practical questions remain: robustness across accents, noisy reference audio, and long-form generation. The paper’s framing is a useful signal, though the details of the evaluation protocol will matter more than the headline.
The jump from VALL-E’s 3-second speaker prompt to full human parity in VALL-E 2 is genuinely striking, especially since it builds on a neural codec language model rather than a traditional pipeline. What interests me most is how this kind of zero-shot replication could translate into creative tooling — I’ve been using Burning Bridges AI for quick meme-style video generation, and a human-parity voice clone would make that workflow far more expressive. Curious whether the paper addresses watermarking or misuse safeguards, since that matters a lot for tools like this.
The jump from VALL-E’s 3-second speaker prompt to VALL-E 2 reaching human parity in zero-shot TTS is impressive, especially since the team had to manage repetition and robustness issues that typically plague codec language models. As someone who plays with generative audio for fun, I’ve been experimenting with The ZombieTrend Generator for quick portrait-to-video clips, and seeing this kind of voice cloning quality makes me wonder how much more convincing those viral zombie scenes could get.
The jump from VALL-E’s 3-second speaker prompt to claimed human parity in VALL-E 2 is striking, especially since they’re still leaning on a neural codec language model rather than a wholesale architecture swap. Voice cloning that convincing makes the meme side of things more interesting too — I’ve been using https://tinydeskaimeme.org/ to drop up to five people into the Tiny Desk band scene, and a crisp synthetic voiceover would slot right in. Curious whether the human-parity claim holds up on accented or noisy reference clips.