PolyAI Launches Dialog-RSN-1 Model

PolyAI today introduced Dialog-RSN-1, a dialog model that directly perceives the user's audio and fuses turn-taking, speech recognition, function calling, and response into a single audio-native model.

For companies that build on PolyAI, that means dialog agents that hear and handle calls the way great human agents would: catching hesitation, hearing frustration before it escalates, and knowing exactly when to speak.

Dialog-RSN-1 is audio-aware on the input side only. It fuses end-pointing, ASR, and response generation into a single audio-aware LLM, so it can reason directly over the raw audio signal, but it leaves speech generation to a separate TTS system.

Dialog-RSN-1 also runs as an ordinary request-based LLM, not a live audio stream, and is probed on demand. A trigger-happy VAD and a few timers decide when to run it on the audio so far, and the first token of its reply settles whether it should start to speak. The cheap acoustic cues only pick when to probe; the model, with full context, makes the actual turn-taking call.

Dialog-RSN-1 hears the user's tone of voice, senses timing, and can pick up on other non-speech audio cues. It can combine the audio signal with the contextual information in the system prompt and conversation history to understand what is being said.

The model can also be prompted on what it hears, like a caller's accent, mood, or the noise on the line. It uses the conversation and system prompt to sort out what a transcript gets wrong: a mispronounced word made obvious by context, a caller spelling "Matthew with two Ts", or a brand name like "Audibel" that isn't the word "audible". And it handles the audio that trips up cascaded systems, like spotting a human vs. a machine, waiting for the beep, or holding through music.

Dialog-RSN-1 decides when to speak. It's trained to respond quickly when the speaker finishes speaking but not interrupt during pauses.

PolyAI built Dialog-RSN-1 through post-training of open weights multimodal models: supervised finetuning (SFT) and reinforcement finetuning (RFT) on in-house training datasets. The first stage of the training, SFT, trains the model on a dataset of millions of conversations, based on real deployments and synthetic conversations, with expert teacher labels. The subsequent Reinforcement Finetuning pushes it higher still, refining the model's reasoning and instruction following.

Post-training teaches the base model to generate a custom output format, containing all predictions for turn-taking decisions, efficient reasoning, text response, tool calls, knowledge citations, and speech transcriptions.
Turn-taking is the very first token predicted by the model: one of EMPTY (no speech detected yet), ONGOING (user has started but not finished talking), or COMPLETE (the user turn is finished, and the assistant should reply). This is very fast to predict, since it is the very first output token, and the KV-cache is warmed up while the user is speaking.

Concretely, Dialog-RSN-1 is driven by a helper VAD that fires on two triggers: acoustic events and a regular time interval.

Dialog-RSN-1 transcribes the user's speech, but only after it has responded or called a function, and after grounding its action by citing its knowledge sources. Transcription happens last, in parallel to speech generation and the agentic loop.

For this first release, PolyAI focused on English, across both post-training and evaluation. The base model is inherently multilingual, and broader language support is coming.