Sprintbot — agile management with AI

Sprintbot ships with Sage, an assistant that performs real actions from natural language: it creates tasks, edits projects, deletes notes. That means handing a model that occasionally hallucinates the keys to several organizations' production data at once. Three concrete risks: a model mistake corrupting data, a user crossing from one workspace into another, and a long conversation driving token cost with no ceiling.

The answer wasn't an agent framework but a layered pipeline of my own, each layer doing one job of containment. A cheap router —a lite model, plus rules that resolve the common intents without spending an LLM call— classifies intent before the expensive model is touched. A multi-turn agent loop capped at 5 iterations and 50,000 tokens that, once the budget runs out, forces a text-only reply instead of spending more. A registry of 41 tools where every call passes schema validation (Zod), role-based permissions (RBAC), and a resilience wrapper (retries with backoff, circuit breaker, timeout). And an explicit human confirmation before any destructive action.

Building the loop, the resilience layer, and the schema conversion by hand —rather than adopting a framework— is more code of my own to maintain. I chose it deliberately: it gave me full control over the token budget, over the confirmation flow, and a way to fix a real mismatch between the JSON Schema that Zod emits (draft-04) and the one Gemini's API expects (draft 2020-12). The cost is maintenance; the gain is that no piece is a black box when something breaks in production.

Sage is live in production. The most telling part wasn't that it works, but what I found when I audited myself: the agent endpoint authenticated the user but never verified they belonged to the workspace they asked for —and because the tools run with admin privileges that bypass Firestore rules, that opened the door to crossing data between organizations. I closed it, wrote the threat model into the commit itself, and logged the rest of the audit with per-finding traceability: what was fixed, what was deferred as known debt, and its phased migration plan. Documenting the risk that remains instead of pretending it doesn't exist is, to me, the difference between building with AI and building with judgment.