This is an intensive, build-everything week. By Friday you'll have written a general-purpose agent that runs in your terminal, reasons with a model, and acts on your machine — reading and writing files, searching the web, running shell commands — while staying affordable on long conversations and pausing for your approval before it does anything irreversible. And you'll have built the evals that prove it works.
We build it in nine short parts. Each adds one capability to the same growing codebase, and each is a runnable checkpoint. No agent framework — just a model API, a loop, and code you understand end to end, because the whole point is to see what the frameworks hide.
A chatbot answers. An agent acts, checks the result, and acts again. This week is about everything that has to be true for that loop to survive contact with the real world.
The walkthrough
Each part adds one capability to the same growing codebase, and each is a runnable checkpoint.
- 01One tool callDescribe a tool to the model, read back a structured call, run it, and return the result. The whole field is built on this exchange.
- 02Single-turn evalsBefore adding features, add a scorer. A tiny eval harness that checks whether the model picked the right tool for a prompt.
- 03The agent loopWrap the exchange in a loop with a step limit and a stop rule: no tool call means the agent is done and should answer.
- 04Multi-turn evalsScore whole trajectories, not just one call — including a negative case where the agent should use no tools at all.
- 05Filesystem toolsRead, write and edit files safely, with errors returned to the model as data it can recover from.
- 06Structured outputs & schemasConstrain the model to a JSON schema so downstream code can trust the shape of what comes back.
- 07Web searchGive the agent access to information it was never trained on, and learn when not to reach for it.
- 08Context managementLong conversations get expensive and eventually overflow. Compact old turns without losing the decision the next step depends on.
- 09Shell tool & human approvalThe most dangerous tool, gated behind a y/n prompt — and a denial fed back to the model as an ordinary observation.
After this week you can
- Explain exactly what an agent loop is and write one without a framework
- Design tool schemas a model can actually use correctly
- Score an agent with automated evals instead of vibes
- Gate irreversible actions behind human approval