On-premises voice AI
Audio, transcripts and model weights stay on hardware you own. No third-party API sits in the call path.
MethodThese are the targets the reference architecture is designed against, measured on our own rack. They are not results from a named client deployment. Your figures depend on GPU class, model pair and how much tool calling sits in the turn; we agree the budget before we build and we report against it.
FIG. 1
A voice turn touches seven systems. In an on-premises deployment every one of them runs on your side of the firewall. So the question a security reviewer actually asks, what leaves the building, has a one-word answer.
| Item | Leaves | |
|---|---|---|
| Call audio | No | Terminates at your SBC |
| Transcripts | No | Written to your storage |
| Model weights | No | Mirrored into your registry |
| Prompts and tool calls | No | Stay on the internal network |
| Model downloads | Once | Once, at install. Air-gapped after. |
MethodReference architecture for a voice agent inside a bank DMZ. Zone boundaries drawn at the SBC and the inference subnet. The only inbound path is the carrier SIP trunk; the only outbound path is the one-time model pull, which can be satisfied from removable media.
TAB. 1
Hardware questions decide the budget, so they come before the demo. These are the figures we size against when we specify a node.
| GPU | ASR / TTS pair | Concurrent calls | Turn latency target |
|---|---|---|---|
| A100 80GB | parakeet / xtts | 40-45 | < 800 ms |
| L40S 48GB | parakeet / xtts | 28-32 | < 850 ms |
| A10G 24GB | whisper-s / piper | 10-14 | < 1000 ms |
| RTX 4090 24GB | whisper-s / piper | 8-12 | < 1000 ms |
MethodSizing model, not a client benchmark. Assumes a 7B-14B open-weights model served under vLLM, 8 kHz G.711 ingress, and one tool call per turn. Concurrency is the point at which p95 turn latency crosses the target, not the point at which the node saturates. We re-measure on your hardware during the pilot and hand you the harness.
TAB. 2
Roughly one in five US households speaks Spanish at home, and a large share of inbound volume switches language mid-call. Most voice stacks treat that as an error condition. We treat it as the default case, because on a collections or benefits line it is.
| Language | Status | Note |
|---|---|---|
| English (US) | Production | Primary target. Tuned per deployment on your own call recordings. |
| Spanish (US / LatAm) | Production | Full turn handling, not a transfer-to-agent fallback. |
| Code-switched EN/ES | Production | Language detected per utterance, not per call. |
| French (Canadian) | Supported | Deployed. Expect a tuning pass on domain vocabulary. |
| Mandarin, Vietnamese, Tagalog | Supported | Deployed for public-sector and health lines. |
| Others | On request | We will tell you when the available models are not good enough. |
MethodStatus reflects what we have run in a production call flow, not what a model card claims. Word error rate varies with domain vocabulary and line quality, so we benchmark on your own recordings during the pilot and give you the result, including any case where a hosted alternative beats our stack.
Engagements
Scoped so that the cheap thing comes first and nothing depends on a long commitment made before you have evidence.
“Is this worth doing at all for our volume?”
We look at your call volume, your flows and your existing telephony, then give you a written recommendation and a hardware cost model. If the answer is that a hosted API is cheaper for you, that is what the document says.
“Can it actually hold a conversation on our calls?”
One narrow call flow, running on one of your GPU nodes, taking real calls. Scoped so that failure is cheap and visible. You keep everything built, including the benchmark harness.
“How do we get this into production properly?”
Telephony integration, CRM and core system wiring, escalation paths, failure modes, load testing and tail-latency tuning. Ends with a handover your platform team can operate without us.
“Who keeps it running?”
Model updates, drift monitoring, tuning as your call mix changes, and an escalation path. Optional by design. The system is built so you can decline this and still run it.
Stack
Set as text rather than a wall of logos. If you want to verify what we deploy before you speak to us, this is enough to do it.
MethodNamed because it is checkable. Every component here is open source or self-hostable under a license that permits commercial deployment, which is what makes an air-gapped install possible and what stops you being locked to us.
Method of work
We pick the call flows worth automating and fix the latency target. Every engineering decision afterwards is argued against that number, and we report against it rather than around it.
Artefact
Written budget, agreed before build
A narrow deployment on a small share of live traffic. The point is to find out early where the stack breaks on your call mix, your accents and your line quality.
Artefact
Working agent, one flow, real calls
Telephony, CRM, escalation and the cases where the agent should hand off to a person. We tune until the tail behaves, not until the average looks good.
Artefact
Load test results, failure-mode notes
Your platform team operates it. We document the failure modes we hit and how we fixed them, including the ones that were our fault.
Artefact
Runbooks, dashboards, training
Against our own interest
On-premises voice AI has a break-even point. Below it, a hosted API is cheaper and you should use one. Here is the arithmetic we use, so you can run it yourself before you call.
Under roughly 50-60 sustained concurrent calls, and with no regulatory requirement forcing data residency, the hosted API usually wins on total cost. We will say so.
Compliance
Most vendors publish a badge row. The useful thing is to separate obligations that bind us directly, from ones that reach us through your contract, from ones where we only supply the technical means for you to meet your own duty.
Ribonic does not currently hold a SOC 2 Type II attestation. We would rather say that than imply otherwise, and we will complete your security questionnaire in writing without a sales call. For an on-premises deployment the honest answer to much of it is that the data never leaves your rack.
Contact
A technical call, not a discovery call. Bring your volumes, your telephony and your constraints, and we will tell you whether this is worth doing, including when it is not.