Agency Brain
A WhatsApp assistant that answers students in French, Arabic and Darija for a study-abroad agency — and knows when not to call the LLM.
The problem
Students message the agency on WhatsApp all day, in three languages and a romanized dialect. Routing rested on two cheap heuristics: a keyword regex for greetings and one similarity score for “is this in scope”. Both broke on real messages: “slm” slipped past the regex, and Arabic questions scored too low and got handed off.
What I built
Two small local classifiers replace the two heuristics. Their output is locked to a fixed answer at generation time, so the router always gets a decision it can act on. Gemini only runs once a message has passed both gates.
- WhatsApp message
- Intent gategreeting only → free welcome
- Scope gatenot answerable → human, free
- Gemini answerthe only paid step
The thinking
The decision that mattered wasn’t which model writes the answers. It was seeing that routing doesn’t need generation at all. Treating “should this reach the LLM?” as a classification problem, and checking the retrieved evidence before paying for an answer, moved accuracy and cost in the same direction.
What I learned
On a yes/no gate, constrained decoding changed nothing I could measure: the model already answered cleanly. Its payoff shows up the moment the output has structure. Free decoding produced usable JSON 39% of the time; constrained decoding, 100%, with no parsing code.
What I’d do differently
The scope gate takes about 7 s per uncertain message on CPU. That’s fine at WhatsApp pace, but a smaller distilled judge would cut it. I’d also fold the hand-written “active conversation” rule into the classifier, so one decision covers it.
Numbers
82% → 89%
routing accuracy, old rules vs. classifiers
28 hand-labeled messages in Darija, English, French and Arabic; same retrieval for both.
39% → 100%
structured outputs usable as-is
Same model and prompt, with and without constrained decoding.
50%
of messages never reach Gemini
Measured traffic mix — 30% small talk, 20% out of scope.