Multi-agent Chatbot (CHEBot)

Context

The hard part of a teaching chatbot isn't the first reply — it's the thirtieth. Single-prompt bots drift: they lose the thread, repeat themselves, race ahead of a confused student or stall out on one they've already lost. CHE's Voyage content lives online as linear, non-interactive videos — the same material in the same order for everyone, with a person administering the program around it. My job was to take that fixed content and turn it into a live lesson that adapts to the person in front of it, with no one in the loop — and have it hold together across a full session.

My bet was that the failure mode wasn't the model, it was the architecture. Asking one prompt to simultaneously plan a lesson, track what the student understands, manage pacing, and sound human is asking it to do four jobs at once, and it does all four badly. So I split them up.

My role

Product and hands-on engineering, and in practice the only person on it. I designed the agent architecture, wrote every agent prompt from scratch, built the backend and the chat front end, built the evaluation and debugging tools, set up hosting and transcript capture, and ran the beta program. I owned it end to end and handed it off clean.

The hardest part was the design judgment — deciding how many agents there should be, what each one was allowed to touch, and what state was permitted to cross between them. Every boundary I drew was a place coordination could fail, so I drew as few as I could get away with.

Approach

I treated the lesson less like a script to recite and more like a team running a session, where each member has one job and stays out of the others' way.

• Four agents, one job each. I orchestrated them with LangGraph, each with a clear lane. Intention is the strategist — it reads signals from the others and decides what should happen next. Comprehension keeps a living "user map" of what the student understands and how they learn, updating quietly in the background. Pacing watches the last few turns and only speaks up when something's off — too fast, too slow, or a good moment to go deeper — and stays silent when things are flowing. Scripting is pure voice: it takes Intention's instruction and makes it sound natural, with zero strategic autonomy of its own. Narrow roles are what let each agent be reliable across a 50+ turn conversation.
• One owner for the lesson plan. The plan lives as a JSON "itinerary" that only the Intention agent can read or edit. Earlier it was a fixed path every agent deferred to; making it a single-owner, changeable guide is what stopped the agents from fighting over it. That decision killed a whole class of coordination bugs.
• Match the model to the job. Each agent runs the model its role actually needs — a stronger reasoning model for strategy, cheaper and faster ones for the background trackers, a strong conversational model for the voice. A full 30-minute run lands around 55–75¢, and chasing that number down (async itinerary edits, streaming the reply token-by-token, running Comprehension every other turn) was a real part of the build.
• Make the behavior visible. I built an in-app evaluation panel — per-criterion scoring, a flag-for-review with notes, live cost counter, and a turn-by-turn view of what every agent was thinking — and leaned on LangSmith to trace what was happening inside the graph. That tooling is what turned "the bot feels off" into a specific prompt I could fix, and it drove the whole test-evaluate-iterate loop.

Outcomes

CHEBot reached a stable beta and went out to external testers, including a cohort of ten people running the 30-Day Voyage, on a real-time Python/FastAPI/WebSocket backend I hardened for long-lived connections. It holds a coherent lesson past 50+ turns, and it adds a layer of personalization the static content never had — grounding the lesson in examples the student raised earlier in the conversation. It's since been integrated into the BeYou product line as the obvious missing piece. It's in the product line, not the final release — no public usage metrics yet.

Reflection

Multi-agent isn't free. Every agent boundary is a seam where coordination can tear, and most of my debugging was at those seams — agents over-reacting to a single turn, or two of them both trying to steer. The wins came from discipline, not cleverness: give each agent a job narrow enough to do reliably, hand any shared state to exactly one owner, and tell the others to stay quiet unless they have real evidence to act on.

The thing I'd carry forward is that splitting one hard prompt into four easy ones only works if you're ruthless about what crosses between them. The architecture isn't the agents — it's the rules about what they're allowed to know about each other. Get those right and the conversation coheres; get them wrong and you're patching contradictions forever.