The gap between a demo and a production AI system is measured in observability, failure handling, cost controls, and the people who have to keep it running at 2am. Most AI projects fail not because the model is wrong but because the infrastructure around it was not built for production from the start.
What production means in practice
A production AI system has four properties a demo lacks: it handles the inputs it was not designed for without crashing, it logs enough information to debug failures after the fact, it has a circuit breaker that prevents a degraded model from affecting your users indefinitely, and someone knows what to do when it breaks.
RAG systems fail in ways that are hard to detect without measurement. Retrieval quality degrades as documents change. Embedding models drift relative to query patterns. Chunk sizes that were optimal at deployment become suboptimal as the document corpus grows. Without automated evaluation running continuously, you will not know retrieval quality is degrading until users tell you.
Cost management
API-based inference costs are a function of token count multiplied by call volume. Systems that looked cheap in testing become expensive at production traffic levels. Cost per query should be a tracked metric from day one, with alerts when it exceeds a defined threshold. The cheapest path is usually a smaller model for the majority of queries with a larger model for fallback -- not the largest model for everything.
Every AI system we deploy includes a cost dashboard and an alert threshold. The team operating it should know cost per query before they know anything else.