AI assistants are setting your defaults

AI coding assistants are quietly deciding the default stack, and teams won't hit the limits until production. Ask one of these tools where to put the data and it says Postgres; ask how to automate something and it says an agent. Builders report that Claude and its peers default that way not because the assistant weighed the workload but because the training corpus leaned that way. Which is mostly fine, and it demos well. But somebody picked the stack before anyone looked at the shape of the data. The bill arrives later, when self-hosted stacks run into infrastructure limits that one Postgres box and a stateless agent were never built for, when retrieval turns out to have no sense of time, and when tokenization drifts because the model updated on its own schedule. Agent adoption looks like big data a decade ago: plenty of executive enthusiasm, plenty of talk, modest real work. The stack mismatch lands when usage grows.

Top Stories#

AI coding assistants push Postgres and agents because the training data did. Developers report that Claude defaults to Postgres for storage and reaches for agent patterns for automation, whatever the workload looks like. That is corpus frequency, not reasoning: Postgres fills the code these models learned from, so Postgres comes out. Teams take the suggestion because it is confident and quick. And sure, it works at prototype scale. Everything works at prototype scale. The choice turns load-bearing later, when the data grows or the access pattern looks nothing like rows and columns.

Self-hosting LLMs revealed the real bottleneck is infrastructure assumptions. A year of running self-hosted LLM stacks turned up something dull and useful: the GPU was rarely the thing holding it back, and the assumptions underneath it were. Single-node databases, request handlers that keep no state between calls, context windows sized once and never revisited. All of it holds at demo scale and stops when agents call each other and one request quietly becomes many. You cannot buy your way out of that, which is awkward for anyone who already budgeted for hardware.

Inference tooling is splitting by workload type. OpenAI described how it keeps voice latency low by holding session state at the edge, near the user, and sending decoded audio to the models rather than making a model one more participant on the call. TokenSpeed, a new open-source engine, targets agent workloads, and Gemma 4 uses multi-token prediction, guessing several tokens ahead instead of one at a time. Interactive inference wants the first syllable out fast; agentic inference wants a lot of tool calls finished cheaply. Those two do not obviously want the same machine much longer.

Retrieval assumptions break in production when time and tokenization drift. A developer bolted a temporal layer onto RAG, the pattern where an app looks up documents and hands them to the model, after noticing the lookup has no idea when anything was written. Ask about last month and it returns a two-year-old document that is, semantically, a lovely match. Separately, tokenization drift: models chop text into tokens, the chopping changes when the model updates, and embeddings indexed under the old chopping stop lining up. Both were fine in the prototype. Now someone has to redo them on time nobody budgeted.

Top Stories#

Top Links#