What are the parts of an AI phone receptionist?
A working receptionist is several systems joined together, and each one can fail on its own. Before choosing tools, list the parts and decide who will build and support each one once real customers are calling. The list below is the minimum for a receptionist that answers, understands, acts on the call and reports what happened. Most of the effort sits in the joins between these parts, such as passing the caller's details from the speech model to the booking tool.
- A phone number and voice carrier account that can route calls to your application
- Call control: answering, transferring and ending calls
- Real-time audio streaming between the call and your application
- A speech model, or separate transcription, language and voice models
- Tools that read calendars and create bookings, messages or records
- Logging, call summaries and an admin screen for updating business information
How does call audio reach the model?
The carrier has to hand you the caller's audio as it happens and play your replies back. Twilio's Media Streams, for example, sends the raw audio of a call to a WebSocket address you specify. A unidirectional stream only lets your application listen; a bidirectional stream, started with the Connect and Stream instructions, also lets it send audio back into the call. On the model side, OpenAI's Realtime API is designed for voice agents, handling turns, tool calls and interruptions within one session, over WebRTC in browsers or WebSocket on servers. Your code sits between the two.
Why does a demo sound better than a live phone line?
Demos are usually run by the person who built them, in a quiet room, saying things the system was designed for. Real callers ring from building sites and cars, talk over the greeting, pause mid-sentence, spell surnames, read out long addresses and ask questions nobody planned for. Each of these needs handling: when to treat silence as the end of a turn, how to confirm a phone number digit by digit and what to do when the model is unsure. Plan to spend more time on these cases than on getting the first call working.
What has to happen after the first call works?
A live receptionist needs ongoing work. Prompts and business information need updating when hours, prices or staff change. Tool calls fail when a calendar is slow or a record is locked, and each failure needs a truthful fallback for the caller. Transfers to staff need re-testing whenever phones change. Someone has to watch for outages and be reachable when the line stops answering at night. Model and API updates from your providers need testing before you adopt them, and usage costs need tracking per call so a spike is noticed quickly.
What are you responsible for if you build it?
A business that builds its own receptionist carries every obligation a provider would otherwise share. The OAIC's guidance on commercially available AI products asks businesses to check how a product handles data, including training data sources, security risks and developer access to their data, and to update privacy policies and notices to cover AI use. The Guidance for AI adoption describes a chatbot as low risk when it answers simple questions during business hours with staff monitoring it, and higher risk when it runs around the clock without oversight. A phone receptionist that answers at night is in the second group, so plan monitoring to match.
When does building your own make sense?
Building suits a business that already employs developers who can support a phone line, has an unusual system to connect that no provider supports, or needs control over where every piece of call data goes. It can also suit a developer who wants to learn the technology on a low-stakes line, such as internal calls. For a small business without that capacity, a managed service moves the support and testing work to the provider, and you can check that in the contract. Compare the total effort as well as the subscription cost.