The AI You See Online Isn't the AI That Ships

Your feed is full of two kinds of AI content. Agents that supposedly run entire companies while the founder sleeps, and screenshots of chatbots telling customers to glue cheese to pizza. One is marketing for AI. The other is marketing against it. Neither one tells you anything about what AI looks like when it actually ships.

I run a consultancy that builds software for mid-market companies, so I spend my weeks inside real codebases and real P&Ls. The AI that survives contact with production looks nothing like either side of your feed. It is narrow. It is boring. And it is quietly doing real work.

What Boring Actually Looks Like

A model reads inbound support tickets and routes them with a confidence score, and anything below the line goes to a human. A pipeline turns meeting transcripts into structured CRM updates that a person approves in one click. A system drafts the first pass of a proposal from past project data, and an account lead edits it. No autonomy theater. A model doing one or two things, wrapped in ordinary software, with a human or a deterministic check at every point that matters.

That gap between demo AI and production AI is not a small one, and it is where most of the budget dies. A demo has to work once, in front of a friendly audience, on a happy path someone rehearsed. Production has to work on the ten thousandth weird input, at 2am, when nobody is watching and the customer on the other end does not care how impressive the launch video was. Every couple years a technology shows up promising to kill all the incumbents and solve all your problems. The companies that get burned are the ones that confuse the launch video for the product.

The Same Failure Pattern, Over and Over

We see it every time we get called in to fix these projects. A team watches the demos, gets excited, and gives the model the whole job instead of a job. They skip the boring parts — the evals, the monitoring, the cost tracking, the fallback paths — because none of that was in the video. Then it hits production and they discover they cannot answer basic questions.

Their APM stack has no idea what a token is. Nobody can debug a bad output because nobody can see one. So the feature gets quietly turned off, and the executive team concludes AI was hype. The model was almost never the problem. The engineering around it was. That is the part nobody posts about, because observability dashboards and eval suites do not go viral. But it is the difference between the 80% of AI initiatives that stall and the ones that compound.

We Have Seen This Movie Before

There is a useful analogy from the last platform shift. In the early cloud days, the winners were not the companies with the most exotic architectures. They were the ones that brought ordinary operational discipline to a new platform: monitoring, cost controls, deployment pipelines, boring stuff. The companies that treated the cloud as magic got the famous six-figure surprise bills.

We are rerunning that movie with AI right now, except the bill can show up faster, because a model can decide on its own how much compute to burn on any given request. That last point deserves its own article, because the economics of AI in production break the mental models most finance teams are using, and I think it catches a lot of companies off guard over the next couple years. That one is next.

The Four Questions That Actually Matter

When you are evaluating AI — for your product or your operations — ignore both halves of your feed. The demos overstate what ships, and the failure screenshots overstate the risk of shipping. Ask the boring questions instead.

The companies getting real value from AI are not the ones with the flashiest agents. They are the ones that could answer those four questions before they shipped.

The moat is not access to the model. Everyone has the model. The moat is the discipline around it.