Inworld Realtime Realtime Voice AI: API Architecture, Capabilities & Integrations
A realtime speech-to-speech platform providing first‑party Realtime TTS-2 and STT, an LLM Router for multi-provider model routing, realtime WebSocket/WebRTC APIs, voice cloning, and providerData runtime extensions.
Peak Demand evaluates the platform in the context of telephony, APIs, business rules, integrations, QA, monitoring, and the operating environment around the agent.
Discuss a Inworld Realtime DeploymentWhat Is Inworld Realtime?
Inworld Realtime is a core realtime speech-to-speech platform: first‑party Realtime TTS-2 and STT, full‑duplex WebSocket/WebRTC realtime APIs, an LLM Router for unified multi‑provider routing, voice cloning, and providerData runtime controls.
Inworld Realtime Platform Profile
Inworld Realtime
Model/platform • Realtime voice stack
Where Inworld Realtime Fits in a Voice AI Technology Stack
Core component — use Inworld Realtime when your product needs low-latency, full‑duplex speech-to-speech with programmatic voice management and unified, runtime-swappable STT/LLM providers. It supplies the realtime media path and model routing; telephony/PSTN, IVR queueing, and telephony primitives were not found in the reviewed official docs and would be addressed by external telephony systems or a custom integration layer.
A Typical Inworld Realtime Production Architecture
The exact architecture depends on the business environment, but Peak Demand evaluates the platform as one layer inside a connected production system.
Inworld RealtimeRealtime STT / LLM / TTS stack for building voice agentsInworld Realtime Capabilities Relevant to Production Voice AI
| Capability | Current position | Scope | Implementation context |
|---|---|---|---|
| Inbound calling | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Outbound calling | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Telephony / phone routing | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| SIP / trunking | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Webhooks / callbacks | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Public APIs | Established | Product-native | Comprehensive public HTTP and realtime (WebSocket/WebRTC) APIs for Realtime, TTS, STT, Router, and Voices are documented (authentication, endpoints, schemas, examples). |
| SDKs / developer libraries | Limited / conditional | Platform-family | Inworld publishes an official CLI (npm @inworld/cli) and language examples (JS/Python) and the docs include example code and a built-in MCP server; no broad, fully-documented multi-language SDK package surface beyond the CLI is explicitly documented in the reviewed pages. |
| Tool / function calls | Established | Product-native | Realtime session supports registering tools (function-like items) in session.update and the realtime event stream includes function call argument events; providerData.auto_tool_response controls follow-up behavior. |
| Transfers / forwarding / handoff | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Conference / queue primitives | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Appointment booking | Not found in reviewed official docs | External integration | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Calendar integration | Not found in reviewed official docs | External integration | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Knowledge bases / retrieval | Limited / conditional | Platform-family | Platform provides memory (automatic conversation memory and summarization) via providerData.memory and the Router/chat completion API supports web-grounded answers (extra_body.web_search); no dedicated hosted KB/retrieval product surface was documented in the reviewed pages. |
| Workflow automation | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Integrations / connectors | Established | Product-native | Realtime STT exposes a unified API to multiple third-party STT providers (Groq, AssemblyAI, Soniox, Deepgram) and Router supports many LLM providers; the platform unifies authentication and request schemas for multi-provider use. |
| Call recording | Not found in reviewed official docs | External integration | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Transcription / speech-to-text | Established | Product-native | Realtime STT offers first‑party inworld/inworld-stt-1 and unified support for third-party providers, with both sync (file) and real-time (WebSocket) streaming endpoints and turn-detection configuration. |
| Text-to-speech / voices | Established | Product-native | Realtime TTS-2 (and TTS-2 Flash) are first‑party realtime TTS models (200+ languages, instant voice cloning, steering); synchronous and streaming TTS REST endpoints and Realtime TTS in the Realtime API are documented. |
| Realtime audio / media streaming | Established | Product-native | Realtime API supports full-duplex audio over WebSocket or WebRTC with sub-second end-to-end speech-to-speech latency and an OpenAI-compatible realtime event schema (with Inworld extensions). |
| DTMF / speech gather | Not found in reviewed official docs | Not applicable / unresolved | Not found in the official documentation reviewed for this profile; this is not a claim that the capability is unsupported. |
| Call logs / analytics / observability | Limited / conditional | Platform-family | API responses include usage metrics fields (usage in STT/TTS responses) and the platform describes continuous monitoring and audit trails in security docs; a centralized call-analytics product is not documented in the reviewed pages. |
| Testing / simulation | Established | Product-native | TTS Playground, WebSocket/WebRTC quickstarts, CLI (MCP server) and live examples are provided for testing and iteration in the docs and portal. |
| Language support | Established | Product-native | Inworld STT (inworld/inworld-stt-1) documents support for 30 STT languages; Realtime TTS-2 documents support for 200+ languages/locales and cross-lingual cloning/steering features. |
| Security / compliance | Established | Platform-family | Platform security claims include SOC 2 Type II, HIPAA & GDPR compliance, zero-data-retention options, AES/TLS encryption, zero-trust architecture, and enterprise SSO (SAML/OIDC) per security pages. |
| Pricing / billing model | Established | Platform-family | Public pricing page documents tiered plans (On-Demand, Creator, Builder, Developer, Growth, Enterprise) with per-1M-character TTS rates, STT per-hour pricing, plan credits, volume discounts, and enterprise custom pricing paths. |
Capabilities marked “Not found in reviewed official docs” were not located in the official documentation corpus reviewed for this profile; that status does not mean the capability is unsupported.
How Inworld Realtime Can Connect to Business Systems
Docs describe a unified multi-provider STT surface (inworld + Groq, AssemblyAI, Soniox, Deepgram) and a Router API that centralizes LLM model selection and conditional routing. ProviderData realtime extensions let you adjust STT, TTS, memory, backchannel and responsiveness at runtime without changing the realtime protocol. Voice cloning and management are exposed as Voices API endpoints.
Realtime STT is a unified multi-provider surface (Inworld + Groq, AssemblyAI, Soniox, Deepgram) enabling provider swap without changing client integration (docs: STT providers) — see source 4.
LLM Router centralizes model selection and conditional routing; routers can reference provider-prefixed model IDs or an inworld router resource (source 19, 23).
Realtime session providerData allows hot-swappable branches (stt, tts, memory, backchannel, responsiveness) to tune runtime behavior without a new protocol (source 24).
Voice cloning and voice management are exposed via Voices API endpoints (clone, update) allowing programmatic voice creation for TTS (sources 15, 22).
Common Inworld Realtime Use Cases
The documented realtime flow is STT → Router/LLM → TTS in a full‑duplex WebSocket or WebRTC session. Realtime event schema (OpenAI-compatible with Inworld extensions) and session.update mechanisms support registering tools, function‑style calls, and providerData controls for turn detection, memory, and backchannel behaviors.
Voice agents (conversational assistants) with realtime STT, LLM reasoning, and expressive TTS responses
Multilingual companions and localized voice experiences using Realtime TTS-2
Agentic workflows that call external tools/functions via Realtime session tools and function calls
Where Inworld Realtime May Be Particularly Strong
Documented strengths in the reviewed corpus:
Strength 1
End-to-end realtime speech-to-speech with sub-second latency and full-duplex WebSocket/WebRTC (STT→LLM→TTS)
Strength 2
High-quality first-party Realtime TTS-2 (200+ languages, instant cloning, steering)
Strength 3
Unified, multi-provider STT and LLM routing (single API to swap providers)
Strength 4
Detailed realtime extensions (providerData) for VAD/turn-detection, memory, backchannel, and TTS segmentation
Where Inworld Realtime May Not Be the Best Fit
Documented tradeoffs and constraints in the reviewed corpus:
Consideration 1
No first‑party telephony/PSTN/SIP primitives documented in reviewed corpus—telephony integration must be external
Consideration 2
Some platform primitives (Router, Voices API, CLI) are separate APIs; not all features are locked into a single SDK
Consideration 3
Enterprise features (custom pricing, on‑prem, data residency) require enterprise plan / account engagement
When Peak Demand May Choose Inworld Realtime
Choose Inworld Realtime when you need:
Best-fit pattern 1
Realtime voice agents and voice-first consumer apps requiring low-latency speech-to-speech
Best-fit pattern 2
Multilingual voice experiences and voice cloning across languages
Best-fit pattern 3
Applications that need unified access to multiple STT/LLM providers and runtime model routing
When another platform may deserve a closer look
Evaluate alternatives when 1
Out-of-the-box PSTN telephony/IVR deployments that expect native SIP/trunk or carrier integrations
Evaluate alternatives when 2
Use cases requiring documented native contact-center features (queues, transfers, DTMF) in the product docs reviewed
Security, Data Handling & Compliance Considerations
Security and compliance details are documented on the platform security pages and cite SOC 2 Type II, HIPAA & GDPR options, AES/TLS encryption, zero-data-retention account options, and enterprise SSO (SAML/OIDC). The docs also recommend short-lived JWTs for client-side realtime keys and caution against embedding long-lived API credentials in clients.
Platform security claims include SOC 2 Type II certification, GDPR & HIPAA compliance options, Zero-Data-Retention account options, AES encryption for data at rest, and TLS for transit; enterprise SSO (SAML/OIDC) is supported (source 7).
Docs and API references emphasize using short-lived JWTs for client-side realtime keys and caution against embedding Basic API credentials in client-side code (authentication docs) (source 2).
How Inworld Realtime Pricing Should Be Evaluated
Public pricing pages document tiered plans (On‑Demand → Enterprise) with per-1M-character TTS pricing, per-hour STT pricing, plan credits, volume discounts, and enterprise custom pricing paths. Realtime TTS-2 and TTS-2 Flash appear on the pricing matrix with unit rates; sales engagement is referenced for custom enterprise terms.
Pricing page documents tiered subscription plans (On-Demand → Enterprise) with per-1M-character TTS pricing and per-hour STT pricing; Growth and Enterprise tiers surface lower unit rates and additional features (source 9).
Realtime TTS-2 / TTS-2 Flash per-1M-character rates, STT per-hour rates, and plan credits/discounts are shown; enterprise pricing and custom TTS/limits are available via contact sales (sources 9, 1, 6).
Testing the Platform Before Production
Docs and the portal provide testing surfaces including a TTS Playground, WebSocket/WebRTC quickstarts, and an official CLI (npm @inworld/cli) plus code examples for iteration and local simulation.
What Peak Demand Adds Around Inworld Realtime
Implementation notes from the reviewed docs: the Realtime API supports WebSocket and WebRTC full‑duplex audio and an OpenAI-compatible event schema with Inworld realtime extensions. Router, Voices (voice clone/update), and the CLI are documented as separate API surfaces—don’t assume all primitives are packaged into a single SDK. The docs show providerData runtime controls and tool/function call events for integrating external tools during sessions. Native telephony/PSTN/SIP/trunking, IVR transfer/queue primitives, DTMF gather, and built‑in calendaring/appointment features were not found in the reviewed official docs; projects requiring those capabilities should plan for external telephony or calendaring integrations.
Discovery & Platform Fit
Determine whether the platform is actually the right choice for the workflow before building around it.
Conversation & Agent Architecture
Design prompts, flows, variables, tools, validation, escalation and business logic.
Telephony & Realtime Infrastructure
Configure the appropriate phone, SIP, CPaaS or realtime transport layer for the deployment.
Middleware & APIs
Build controlled AWS, Cloudflare, API, webhook or middleware layers where systems require additional validation and orchestration.
Business-System Integration
Connect CRM, scheduling, EMR/EHR, ERP, databases, helpdesk, ordering, field-service or proprietary software where suitable integration surfaces exist.
QA & Managed Operations
Test workflows, monitor production behavior, review failures, measure outcomes and refine the implementation over time.
Inworld Realtime Questions
Does Inworld Realtime provide native PSTN/telephony (SIP/trunk) capabilities?
In the reviewed official documentation, native PSTN/SIP/trunking and telephony primitives (inbound/outbound calling, carrier trunking) were not found. The docs focus on WebSocket/WebRTC realtime media APIs and provider integrations; teams requiring PSTN should plan for an external telephony integration layer.
Can I clone or programmatically manage voices?
Yes — the reviewed docs include a Voices API with endpoints for cloning and updating voices, enabling programmatic creation and management for TTS (source: Voices API docs).
How does Inworld handle multi‑provider STT and LLM selection at runtime?
The platform documents a unified STT surface that can target Inworld or third‑party STT providers (Groq, AssemblyAI, Soniox, Deepgram). Router resources and provider-prefixed model IDs centralize LLM selection and conditional routing; providerData realtime extensions allow runtime tuning of provider behavior without changing the realtime protocol.
What language coverage is available for STT and TTS?
Docs show the first‑party inworld STT model supports roughly 30 STT languages, and Realtime TTS-2 documents support for 200+ languages/locales with cross-lingual cloning and steering features.
Is there an official SDK for multiple languages?
The reviewed documentation documents an official CLI (npm @inworld/cli), language examples (JavaScript/Python), and a built‑in MCP server for testing. The docs do not present a broad, fully documented multi‑language SDK package surface beyond those artifacts in the reviewed pages.
What testing and iteration tools are provided?
The docs and portal provide a TTS Playground, WebSocket/WebRTC quickstarts, CLI-based testing utilities, and live examples for building and iterating on realtime voice flows.
What are the documented security and compliance claims?
Inworld documentation cites SOC 2 Type II, HIPAA & GDPR compliance options, AES/TLS encryption, zero-data-retention options, and enterprise SSO (SAML/OIDC). The docs also emphasize using short‑lived JWTs for client‑side realtime access and avoiding embedding long‑lived credentials in clients.
Explore Retell AI
Retell AI is one of the full-stack Voice AI platforms Peak Demand evaluates for phone-first deployments, custom integrations, telephony, APIs, and managed production workflows.
Explore Retell AIPeak Demand may earn a commission from this link.
Planning a Inworld Realtime Deployment?
Peak Demand can help evaluate platform fit, design the architecture, connect telephony and business systems, implement controlled tools and integrations, test edge cases, and manage the operational layer after launch.
Discuss a Inworld Realtime DeploymentOfficial Inworld Realtime Sources Reviewed
This profile is maintained using official or first-party vendor sources. Current vendor documentation remains the source of truth for an active production decision.
Last researched: 2026-08-19
Next recommended review: 2026-11-17
