Context as a Product Layer

In 2025, the field quietly renamed one of its core skills. We stopped saying "prompt engineering" and started saying "context engineering." Andrej Karpathy pushed the term. Shopify's CEO endorsed it. Within a few months it was just what people called the work.

Renames are a tell. We don't rebrand the things that are working fine. This one happened because the old frame had run out of room. "Prompt engineering" treats the input as a clever sentence you tune. "Context engineering" treats it as something you assemble: what the model knows at the moment it acts, where that knowledge comes from, and what you deliberately leave out. The unit of work got bigger. It stopped being a string and became a system.

I think even the rename undershoots what's happening. Context isn't just a skill we got more serious about. It's becoming a layer in the stack, with its own architecture, its own economics, and its own lifecycle. And most products are still shipping it like a feature.

A feature, or a floor?

Treated as a feature, context shows up as a memory toggle you switch on. A knowledge base you upload documents to. A system prompt someone tunes. A bigger window you pay more for. One more capability in the list, sitting next to search and notifications.

Look at where it actually sits, though, and it doesn't behave like a feature. It sits between the model and the product. The model supplies raw capability. The product is the part the user touches. Context is the floor in between: everything the system knows about this user, this task, this moment, plus everything it's been told about how to behave.

Once you draw it this way, something becomes hard to unsee. The model layer is converging. Everyone can rent a frontier model, and the gap between the best one and the second-best one keeps shrinking. The product layer is mostly table stakes. The decisions that make one AI product feel sharp and another feel generic are migrating into the middle. Into context.

That's the claim here. Context is becoming a product layer, and treating it like a feature is how you end up with the thing that dazzles in the demo and forgets you by Tuesday.

Context isn't free

Here's the assumption almost every team starts with: more context is better. If the model knows more, it'll do better. So load everything. Stuff the window. Retrieve aggressively. When in doubt, add.

The research has been pulling the other way for a while. In 2023, a Stanford-led paper with a title that became a meme, "Lost in the Middle," showed that models use long inputs unevenly. They lean on the beginning and the end and lose the middle. Pile more in, and the material in the soft middle quietly gets ignored. In 2025, the team at Chroma put a sharper name on a broader version of the problem: context rot. Across a range of models, performance on even simple tasks degrades as the input gets longer. Not because the task got harder. Because the context did. Drew Breunig catalogued the specific ways long context backfires, with names worth knowing: poisoning, distraction, confusion, and clash. Extra material doesn't just sit there inert. It pulls the model off course. Anthropic, writing up its own guidance for building agents, said it plainly: context is a finite resource with diminishing returns.

I've seen the same shape from the other side. Looking at a large pile of real sessions from an agent I work on, the thing that fell as conversations got long wasn't the agent's awareness of context. That held steady. It was efficiency. The agent kept noticing everything and got slower and looser at actually using it.

The reason this matters is what it does to the product decision. If more context were free, the right move would be to load all of it, always. Since it isn't free, the job changes. The job becomes choosing. What to carry into this moment, what to leave out, and when to let go of something you were holding. That's not a model setting you turn up. That's a design decision, and it's one of the more important ones in the whole product.

The lazy version of context engineering is "give the model everything." The real version is curation under a budget, same as any other part of a product where attention is scarce. The window is big now. That's exactly why what you choose to put in it matters more, not less.

Not all context is the same asset

The second thing the field keeps relearning is that we lump very different things under one word. When people say "context," they usually mean three things that behave nothing alike.

Durable context is the good stuff. Who this customer is. How they work. What's true about their business that won't change next week. This compounds. The longer the product knows someone, the more of it accrues, and the more valuable it gets. This is the asset.

Capability scaffolding is a different animal. It's the instruction you write to paper over what the model can't quite do on its own yet. The careful prompt that coaxes the right format. The workaround for a weak spot. Useful today, and depreciating fast. As one OpenAI leader put it, models will eat your scaffolding for breakfast. Every release closes some gap you were compensating for, and the clever workaround becomes dead weight you forgot to remove.

Work products are the third pile, and usually the biggest by volume. The scratch files, the data the agent pulls and chews on mid-task, the intermediate junk. Almost none of it is worth keeping. It's exhaust, not memory.

The mistake is treating these as one pile. They have different half-lives. Durable context you protect and invest in. Scaffolding you treat as debt and pay down as the models get stronger. Work products you let evaporate. Notion framed the durable layer about as well as anyone: memory is just pages and databases. Not a magic new primitive bolted onto the model. The domain objects you already have, made available to the agent with the right permissions. The context worth keeping usually already lives somewhere in your product. The work is connecting it, not inventing it.

The layer nobody fills in

The most valuable context is the kind only your customer has. Their history, their preferences, the shape of their particular world. There's a point that gets made often about models that lands harder for context: the durable moat is the workflow signal the frontier labs can't see. It lives with the customer, not the lab.

The catch is that this layer is almost always the one the customer controls, and it's almost always empty. The auto-generated defaults carry the experience. The part a person could shape, the place where they teach the product about their specific situation, sits untouched. Not because it wouldn't help. Because filling it is work, and most products never make that work feel worth it.

That reframes the opportunity. It isn't another context feature or a bigger window. It's making the context you already have easy to give and obviously worth giving. Whoever gets people to actually fill that layer in ends up holding something a competitor can't copy, because they end up holding context no one else has.

Where the moat goes

Step back and the strategic picture is clean.

The model is becoming something you rent, and your competitor rents the same thing. Capability arrives on a schedule set by the labs, and it arrives for everyone at once. You don't win there for long.

The harness is what you build on top, and it's where the differences live. When a team gets a sudden jump in what its AI can do, the cause is usually less a new model than a better loop around the model it already had: the scaffolding, the retrieval, the context. GitHub's engineering leadership put it about as bluntly as anyone. Context is everything. They were not talking about the model. They were talking about the layer above it.

This has hardened into something close to a refrain among people shipping serious AI systems: the harness is the moat, not the model. I'd state it a notch more precisely. The moat is the context you have that no one else does, plus how well you carry it. The first part is proprietary. The second part is craft. Neither shows up in a model release, which is exactly why neither gets copied when the next one ships.

I want to build products that treat context like infrastructure. Architected with the same seriousness you'd give a database schema, not sprinkled on near the end as a feature. Because the two paths really do diverge. One gives you a product that learns a person and gets a little better at knowing them every week. The other gives you a very smart stranger who reintroduces themselves every session.

To be sure, this plays out over quarters, not weeks, and the model layer will keep improving on its own the whole time. That's sort of the point. The model gets better whether or not you do anything. The context layer only gets better if someone designs it.

Context stopped being a feature a while ago. We've just been slow to rename it.