Agentic AI in production: what actually ships

Agent demos are easy. An agent that runs unattended against real customer data for six months is a different engineering problem.

Bound the blast radius Every tool an agent can call is a permission. Treat the tool registry like an IAM policy, not a convenience layer.

• Scope each tool to a single verb and a single resource
• Require an idempotency key on every write tool
• Put irreversible actions behind an explicit human approval step

Plan, then act We separate planning from execution. The model proposes a plan as structured JSON, a deterministic validator checks it against a schema and a policy list, and only then does the executor run steps.

Observability is the product Log the full trace: prompt version, tool inputs, tool outputs, token spend, and latency per step. Without traces you cannot debug an agent — you can only rerun it and hope.

Evaluate continuously Keep a golden set of 100-300 real tasks. Run it on every prompt or model change. If pass rate drops, the change does not ship. Vibes are not a release gate.