Switching an OpenAI Realtime voice agent to ElevenLabs: the 2-line change, the real cost per call, and the WebRTC catch

Disclosure: this post has affiliate links. If you sign up through them I may earn a commission, at no extra cost to you.

TL;DR: ElevenLabs shipped Eleven Realtime on Oct 20, 2026: two speech-to-speech models (v1 Mini and v1 Max) behind an API that speaks OpenAI's Realtime protocol. If your voice agent already uses @openai/agents-realtime over WebSocket, the switch is a new endpoint URL and a new model string. If you're on WebRTC, it's a transport rewrite first. Cost is $0.08 per connected minute plus LLM tokens, and the per-minute part dominates.

What actually shipped

• Eleven Realtime v1 Mini (eleven_realtime_v1_mini): the default. Built for fast, natural conversation: assistants, tutors, characters.
• Eleven Realtime v1 Max (eleven_realtime_v1_max): reasons before it answers. Built for support, lead qualification and multi-step tool calling.
• Both cover 90+ languages and any of the 17,000+ voices in the ElevenLabs library.
• Under the hood it's a cascaded stack (speech recognition, turn-taking, an LLM, and a voice), tuned end to end and exposed as one model ID. The practical upside: every step produces text, so you can see what the agent heard versus what it decided to say.

 Browser mic ──PCM16 24kHz──► wss://api.elevenlabs.io/v1/convai/realtime │ ┌──────────┴───────────┐ │ ASR → turn-taking → │ one model ID: │ LLM → TTS (tuned as │ eleven_realtime_v1_mini │ a single system) │ eleven_realtime_v1_max └──────────┬───────────┘ Speaker ◄──PCM16 audio───────────┘ (+ text at every step for logs/guardrails) 

The numbers ElevenLabs published

Benchmarks are from Artificial Analysis as reported in ElevenLabs' launch post; prices from the ElevenLabs API pricing page (checked Nov 3, 2026).

The one line that should drive your model choice: tool use, 60% vs 87%. If your agent calls functions (book a slot, look up an order), Mini will drop roughly 4 in 10 tasks on that benchmark. If it mostly chats, Mini is the better-sounding, cheaper pick.

The migration: what changes in your code

ElevenLabs' docs say the API uses the same client and server events over the same WebSocket transport as OpenAI's Realtime API. In practice there are two changes in the browser, plus one on your server.

You'll need an ElevenLabs API key for step 1 (create one here; the free tier is enough to test).

1. Server: mint a short-lived client secret (never ship your API key to the browser).

import os from elevenlabs.client import ElevenLabs from elevenlabs import (CreateClientSecretRequest, ExpiresAfter, OpenAiSessionConfigInput, OpenAiAudioConfigInput, OpenAiAudioInputConfigInput, OpenAiAudioInputFormat, OpenAiAudioOutputConfigInput, OpenAiAudioOutputFormat, OpenAiTurnDetection) el = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"]) secret = el.conversational_ai.realtime.create_client_secret( request=CreateClientSecretRequest( session=OpenAiSessionConfigInput( type="realtime", model="eleven_realtime_v1_mini", # or eleven_realtime_v1_max  instructions="You are a concise, friendly voice assistant.", output_modalities=["audio"], audio=OpenAiAudioConfigInput( input=OpenAiAudioInputConfigInput( format=OpenAiAudioInputFormat(type="audio/pcm", rate=24000), turn_detection=OpenAiTurnDetection(type="semantic_vad", eagerness="medium"), ), output=OpenAiAudioOutputConfigInput( format=OpenAiAudioOutputFormat(type="audio/pcm"), voice="brian"), ), ), expires_after=ExpiresAfter(anchor="created_at", seconds=600), ) ) # return only secret.value to the browser 

2. Browser: same OpenAI SDK, new URL and model.

import { OpenAIRealtimeWebSocket, RealtimeAgent, RealtimeSession } from "@openai/agents-realtime"; const { value: clientSecret } = await fetch("/api/token").then(r => r.json()); const agent = new RealtimeAgent({ name: "support", instructions: "Keep replies short." }); const transport = new OpenAIRealtimeWebSocket({ url: "wss://api.elevenlabs.io/v1/convai/realtime", // ← change #1 }); const session = new RealtimeSession(agent, { transport, model: "eleven_realtime_v1_mini", // ← change #2 config: { outputModalities: ["audio"], audio: { output: { voice: "brian" } } }, }); await session.connect({ apiKey: clientSecret }); 

The catches nobody puts in the headline

1. WebSocket only. If your app uses the SDK's WebRTC transport, you have to move to WebSocket first.
2. You handle the audio. The OpenAI SDK only wires mic and speaker automatically over WebRTC. Over WebSocket you capture and play PCM16 at 24 kHz yourself (wavtools works; it's what OpenAI's own WebSocket examples use).
3. Browser flow only, for now. The docs say server-side OpenAI Realtime SDK docs aren't published yet, so phone-bridge setups that run the SDK on a server are untested territory.
4. The API is separate from ElevenAgents. Phone numbers, WhatsApp, knowledge base, guardrails and testing live in ElevenAgents. If you need those, build there and pick a Realtime model from the dropdown instead.
5. Burst pricing. Connected audio above your plan's concurrency limit bills at $0.16/min, double the normal rate. Concurrency runs from 4 calls (free/pay-as-you-go) to 40 (Business).

What a 5-minute call costs (worked estimate)

The connected-audio fee is fixed: 5 × $0.08 = $0.40. Tokens depend on your prompt and turn count. Here's an illustrative support call with my assumptions spelled out: 30 turns, about 3,000 tokens of context re-sent per turn (90k input tokens total), and about 60 output tokens per reply (1.8k output).

Two takeaways:

• On Mini, tokens are noise; minutes are the bill. Hang up faster (end-call tool, tight prompts, no long hold music) and you save more than any prompt golf would.
• On Max, input tokens are the lever. At $1.80/M input, a bloated 3k-token system prompt re-sent every turn is about a quarter of the call cost. Cut it to 800 tokens and the Max call drops toward $0.45.

Copy-paste: a voice-agent system prompt that keeps calls short

You are {brand}'s phone assistant. Goal: resolve the caller's issue in under 3 minutes. Rules: - Answer in 1–2 sentences. Never read lists longer than 3 items aloud. - Ask one question at a time. - If you need data, call the tool immediately; say "one sec" while it runs. - When the issue is resolved, confirm in one sentence and call end_call. - If the caller asks for a human or you fail twice, transfer. 

Which one should you pick?

If you want to try it, the free tier includes roughly 14 minutes of Realtime v1 Mini, enough to run your existing client against it this afternoon. Get an ElevenLabs API key →

Sources: ElevenLabs launch post (elevenlabs.io/blog/realtime), OpenAI Realtime compatibility guide (elevenlabs.io/docs/eleven-agents/guides/openai-realtime-compatibility), ElevenLabs API pricing (elevenlabs.io/pricing/api).

Affiliate disclosure: TokenLean participates in affiliate programs, including ElevenLabs'. Some links in this article are affiliate links, which means I may earn a commission if you click and make a purchase, at no additional cost to you. I only recommend tools I'd use in a real build, and the facts above come from the vendor's public docs and pricing pages. Prices and benchmarks change; check the official pages before you commit.