Best AI Models for LiveKit

Mercury 2 represents a groundbreaking advancement in reasoning models, specifically designed for real-time voice interaction as it can quickly answer phone calls. Unlike traditional autoregressive models that leave callers in silence while generating responses one token at a time, Mercury 2 employs a diffusion large language model architecture capable of producing over 1000 tokens per second with standard NVIDIA GPUs. This remarkable speed allows it to complete a full reasoning process and begin speaking within a timeframe that aligns with natural conversational flow, effectively shortening the typical wait time from several seconds to approximately 300 milliseconds. The operational mechanism of Mercury models involves transforming clear text into noise, after which a conventional Transformer is trained to reverse this transformation and predict the original text across all positions at once. By utilizing a denoising approach that engages multiple tokens simultaneously, generation becomes more efficient, enabling speeds akin to custom silicon on NVIDIA H100s while improving responsiveness in voice applications. As a result, Mercury 2 not only enhances user experience but also sets a new standard for interactive voice technologies.

The Gemini Live API is an advanced preview feature designed to facilitate low-latency, bidirectional interactions through voice and video with the Gemini system. This innovation allows users to engage in conversations that feel natural and human-like, while also enabling them to interrupt the model's responses via voice commands. In addition to handling text inputs, the model is capable of processing audio and video, yielding both text and audio outputs. Recent enhancements include the introduction of two new voice options and support for 30 additional languages, along with the ability to configure the output language as needed. Furthermore, users can adjust image resolution settings (66/256 tokens), decide on turn coverage (whether to send all inputs continuously or only during user speech), and customize interruption preferences. Additional features encompass voice activity detection, new client events for signaling the end of a turn, token count tracking, and a client event for marking the end of the stream. The system also supports text streaming, along with configurable session resumption that retains session data on the server for up to 24 hours, and the capability for extended sessions utilizing a sliding context window for better conversation continuity. Overall, Gemini Live API enhances interaction quality, making it more versatile and user-friendly.

Gemini 3.8 Live is a native speech-to-speech AI model from Google DeepMind designed for low-latency conversational agents and real-time voice applications. The model can reason and execute tasks while maintaining the natural flow of an audio conversation. Its asynchronous function calling capability allows external APIs and tools to run in the background without forcing the agent to stop speaking while it waits for results. Developers can combine streamed audio with structured information through incremental content updates, allowing responses to adapt as new data becomes available. Visual context support enables applications to ground conversations in live images or video so agents can understand both what users say and what they are looking at. Gemini 3.8 Live supports more than 97 languages and is designed to maintain consistent accents across multilingual experiences. The model also emphasizes alphanumeric precision for accurately understanding information such as account identifiers, confirmation codes, technical values, and claim numbers. A related Gemini 3.8 Live Extended Thinking model adds configurable reasoning for more complex, multi-step tasks while continuing to interact with the user. Gemini 3.8 Live is available through the Gemini API, Google AI Studio, and integrations with real-time development platforms such as LiveKit, Pipecat, Agora, LangChain, and Vercel.