v0.1 · open source · MIT

Voice agents that actually do the work.

PJ Voice is an open source AI phone agent. It answers calls, looks up the caller, checks the calendar, and books, moves or cancels real appointments, with correctness enforced in code, not hoped for in a prompt.

$ pip install -e '.[voice]' && pjvoice chat examples/demo-salon.yaml
Watch a call

Tools, stages and guards,
live in one conversation.

A scripted example call to the bundled demo salon. The tool calls, stage switches and guard responses mirror what the runtime and the reference tool server really do.

Demo Salon · +1 555-0100 00:00

Stage graph

triage⇄ booking⇄ manage

Each stage has its own instructions and tool subset. Handoffs are silent, with loop caps.

Server-side session

session_idcall-7f3a caller_phone+15550142 (injected) customer— offered slots0 bookings0

The model never handles customer IDs. The tool server binds identity to the call.

How it works

A streaming pipeline, tuned for the phone.

Callers start hanging up after about a second of silence. Every stage streams, so the agent starts speaking while it's still thinking.

SIP / WebRTC

Phone or browser audio into a LiveKit room.

transport

VAD + turns

Silero VAD and a semantic turn detector decide when you've finished.

endpointing

Speech to text

Streaming transcription with partial results.

STT

LLM + tools

Stage-scoped instructions and tools, with a fallback model.

reasoning · actions

Text to speech

Streams audio sentence by sentence, with repeat suppression.

TTS
What's inside

Everything between "hello"
and a booking in the calendar.

Stage graphs

Split a call into steps like triage → booking → manage. Small prompts, focused tools, silent handoffs, loop caps.

Injected arguments

Mark a parameter x-injected and the model never sees it. It can't attach a booking to the wrong customer.

One brain, many channels

The same config runs phone calls, browser calls, SMS/chat turns and evals, through one shared engine.

Multi-tenant control plane

Phone-number routing, live tenant configs via webhook, and last-known-good fallback, so a tenant outage never drops a call.

Simulated-caller evals

An LLM plays the caller; weighted validators check tools and final state. Catch regressions before your callers do.

Bring any model

LiveKit Inference model strings for STT, LLM and TTS, plus any OpenAI-compatible API in text mode.

Safety by design

Prompts ask. Code enforces.

Eventually the model will invent a time, reuse the wrong ID, or book twice. The reference tool server shows the guards that make that harmless.

01

Offered-slot guard

Only slots this call was actually offered can be booked. Invented times are rejected with "call check_availability first".

02

Server-side identity

The caller is bound to the session_id. Booking, rescheduling and cancelling read it from there, never from the model.

03

Re-check before write

Taken in the meantime? The error carries fresh alternatives, so the agent offers a new time in the same turn.

04

Idempotent bookings

A repeated book_appointment after an interruption returns the existing booking. No duplicates.

05

No-progress nudge

Same call, same arguments, same answer? The runtime tells the model so, instead of letting it loop.

06

Hop caps & fallbacks

Stage ping-pong is refused, a failing LLM falls back to another provider, and tool timeouts become a polite recovery.

Developer experience

A YAML file, an HTTP endpoint, a phone number.

Describe the agent in config. Put business logic behind a tiny HTTP protocol in any language. Test it in your terminal before it ever rings.

  • Plain HTTP tools. One JSON envelope in, text for the model out.
  • Test without a phone. pjvoice chat runs the same engine in text.
  • CI-friendly. Scripted-LLM unit tests need no API keys.
id: trattoria
greeting: Good evening, Trattoria Demo. Book a table?
models: {llm: openai/gpt-4.1-mini, fallback_llm: openai/gpt-4o-mini}
instructions: |
  Ask for date and party size, call check_tables,
  offer two or three times, confirm, then reserve.
tools:
  - name: reserve
    url: ${RESTAURANT_TOOLS_URL}
    parameters:
      type: object
      properties:
        option_id: {type: string}
        name: {type: string}
        caller_phone: {type: string, x-injected: caller_phone}  # hidden from the model
def reserve(s: Session, args: dict) -> str:
    if s.booking_id:                          # idempotent
        return "Already reserved. Do not book again."
    option = s.offered.get(args.get("option_id"))
    if option is None:                        # no invented times
        raise ToolError("Not offered on this call. Call check_tables first.")
    with _lock:                               # re-check before write
        table = free_table(*option)
        if table is None:
            raise ToolError("Just taken. Call check_tables again.")
        ...
    return f"Reserved: {describe(b)}. Read it back to the caller."
name: party of four books Friday evening
persona: >
  You are Sam. You want a table for four this Friday
  around 7 PM. Accept the closest time offered.
validators:
  - {type: tool_called, tool: reserve, weight: 3}
  - {type: transcript_excludes, pattern: "option_id", weight: 1}

$ pjvoice eval assistant.yaml evals/ --tools-app server:app --repeat 3
Free course

Learn to build voice agents.

A practical guide from first principles to production, where every chapter maps to working code in the repo.

Quick start

Talking to an agent in two minutes.

terminal
$ git clone https://github.com/pjsofts/pj-voice && cd pj-voice
$ uv venv && uv pip install -e '.[dev]'
$ export OPENAI_API_KEY=sk-...
$ pjvoice tools &                         # demo booking server on :8100
$ pjvoice chat examples/demo-salon.yaml --show-stage
agent> Thanks for calling Demo Salon. How can I help you today?
you>   I'd like a haircut on Thursday morning
agent> …

# ready for real calls?
$ uv pip install -e '.[voice]' && pjvoice control & pjvoice worker dev

Build the agent your callers won't hang up on.

Open source, MIT licensed, and built to be read. Star it, fork it, ship it.