-
Prompt caching in production: where the savings disappear
KV cache reuse can cut inference costs and latency substantially on paper. Most production deployments capture a fraction of that gap, and the reason is structural rather than a missing configuration flag.
-
Why agent memory degrades in production
What happens to agent memory stores after weeks of production use, and what the read path never exposes.
-
Context engineering as a production discipline
Failure modes, architectural patterns, and evidence from real systems.
-
Why LLM evaluation metrics look stable but customers are unhappy
Classic metrics hide the failures users notice. Production evaluation should measure friction, drift, and task completion.
-
Context limits degrade routing quality faster than generation
Routing and classification under long prompts: score dilution, margin collapse, routing collapse, and practical caps.