PJ Voice is an open source AI phone agent. It answers calls, looks up the caller, checks the calendar, and books, moves or cancels real appointments, with correctness enforced in code, not hoped for in a prompt.
A scripted example call to the bundled demo salon. The tool calls, stage switches and guard responses mirror what the runtime and the reference tool server really do.
Each stage has its own instructions and tool subset. Handoffs are silent, with loop caps.
The model never handles customer IDs. The tool server binds identity to the call.
Callers start hanging up after about a second of silence. Every stage streams, so the agent starts speaking while it's still thinking.
Phone or browser audio into a LiveKit room.
Silero VAD and a semantic turn detector decide when you've finished.
Streaming transcription with partial results.
Stage-scoped instructions and tools, with a fallback model.
Streams audio sentence by sentence, with repeat suppression.
Split a call into steps like triage → booking → manage. Small prompts, focused tools, silent handoffs, loop caps.
Mark a parameter x-injected and the model never sees it. It can't attach a booking to the wrong customer.
The same config runs phone calls, browser calls, SMS/chat turns and evals, through one shared engine.
Phone-number routing, live tenant configs via webhook, and last-known-good fallback, so a tenant outage never drops a call.
An LLM plays the caller; weighted validators check tools and final state. Catch regressions before your callers do.
LiveKit Inference model strings for STT, LLM and TTS, plus any OpenAI-compatible API in text mode.
Eventually the model will invent a time, reuse the wrong ID, or book twice. The reference tool server shows the guards that make that harmless.
Only slots this call was actually offered can be booked. Invented times are rejected with "call check_availability first".
The caller is bound to the session_id. Booking, rescheduling and cancelling read it from there, never from the model.
Taken in the meantime? The error carries fresh alternatives, so the agent offers a new time in the same turn.
A repeated book_appointment after an interruption returns the existing booking. No duplicates.
Same call, same arguments, same answer? The runtime tells the model so, instead of letting it loop.
Stage ping-pong is refused, a failing LLM falls back to another provider, and tool timeouts become a polite recovery.
Describe the agent in config. Put business logic behind a tiny HTTP protocol in any language. Test it in your terminal before it ever rings.
pjvoice chat runs the same engine in text.id: trattoria greeting: Good evening, Trattoria Demo. Book a table? models: {llm: openai/gpt-4.1-mini, fallback_llm: openai/gpt-4o-mini} instructions: | Ask for date and party size, call check_tables, offer two or three times, confirm, then reserve. tools: - name: reserve url: ${RESTAURANT_TOOLS_URL} parameters: type: object properties: option_id: {type: string} name: {type: string} caller_phone: {type: string, x-injected: caller_phone} # hidden from the model
def reserve(s: Session, args: dict) -> str: if s.booking_id: # idempotent return "Already reserved. Do not book again." option = s.offered.get(args.get("option_id")) if option is None: # no invented times raise ToolError("Not offered on this call. Call check_tables first.") with _lock: # re-check before write table = free_table(*option) if table is None: raise ToolError("Just taken. Call check_tables again.") ... return f"Reserved: {describe(b)}. Read it back to the caller."
name: party of four books Friday evening persona: > You are Sam. You want a table for four this Friday around 7 PM. Accept the closest time offered. validators: - {type: tool_called, tool: reserve, weight: 3} - {type: transcript_excludes, pattern: "option_id", weight: 1} $ pjvoice eval assistant.yaml evals/ --tools-app server:app --repeat 3
A practical guide from first principles to production, where every chapter maps to working code in the repo.
The STT → LLM → TTS pipeline and the latency budget.
02VAD, endpointing, interruptions and echo.
03Designing tools for voice; safety on the server.
04Prompts, stages and confirmations for the ear.
05PSTN, SIP trunks, caller ID, transfers, consent.
06Scripted tests, simulated callers, regressions.
07Reliability, observability, cost and privacy.
++Why PJ Voice is built the way it is.
Write a tool server, configure the assistant, chat with it, add evals, and put it on a phone number.
$ git clone https://github.com/pjsofts/pj-voice && cd pj-voice $ uv venv && uv pip install -e '.[dev]' $ export OPENAI_API_KEY=sk-... $ pjvoice tools & # demo booking server on :8100 $ pjvoice chat examples/demo-salon.yaml --show-stage agent> Thanks for calling Demo Salon. How can I help you today? you> I'd like a haircut on Thursday morning agent> … # ready for real calls? $ uv pip install -e '.[voice]' && pjvoice control & pjvoice worker dev
Open source, MIT licensed, and built to be read. Star it, fork it, ship it.