We Don’t Always Need a Bigger Model: We Need Better Context

Working with artificial intelligence in business is changing how I think about the model compared with its context. For a long time, when an answer was not good enough, I assumed we needed a bigger model with stronger reasoning, perhaps from another provider. The model matters, but we often try to solve with scale what is really a problem of information, architecture, and organizational knowledge.

That choice also has an economic consequence. Larger models usually mean higher inference costs, more latency, and more infrastructure. Add huge prompts, many tools, and abundant context, and the cost rises. Across thousands or millions of queries, the question is no longer what one call costs, but what the complete process costs.

We can use one of the most advanced models available and ask it about a product feature, a commercial condition, or a decision made months ago. If the information is outdated, poorly structured, or does not arrive at the right time, the answer will have an obvious limit. We often blame the model even though the problem lies in everything that happens beforehand. An AI does not know our company simply because it is powerful: that knowledge remains scattered across systems, documents, and people.

This is why PIM, ERP, CRM, databases, documentation, APIs, and SharePoint can matter more than the model. The goal is not to squeeze everything into an enormous window, but to identify what each request needs and deliver it reliably. If an answer requires five facts, sending fifty pages may increase cost, latency, and noise without improving it.

This leads me to think less about isolated models and more about complete systems. Before answering, a layer can identify the request, the information it needs, the relevant tools, and the right model. Some queries will justify a powerful generative model; others will only require classifying an input, querying a structured source, or applying a rule. They should not all carry the same cost.

This is where small, specialized models such as JEV or Laya, which we are exploring, fit in. Their role does not have to be writing the final answer. They can act as qualifiers: detecting intent and domain, selecting a tool, deciding what information to retrieve, or determining whether to escalate the request. Instead of asking them for an open-ended answer, we ask them to make a decision within a controlled set of possibilities.

They can still make mistakes, but those errors are easier to contain and measure. If a model must choose between product, pricing, documentation, special requests, or support, we can evaluate its accuracy, analyze false positives, and set confidence thresholds for escalating uncertain cases. We do not eliminate error; we turn an open-ended problem into a more limited and observable one.

This first layer reduces the context received by the generative model. Instead of providing every instruction, tool, and source just in case, we can build an environment for each request. One query might receive a short prompt, a single tool, and only the essential data; another would have a different configuration. The model can remain the same, but the problem is better bounded.

The benefit is not limited to tokens. The more tools, rules, and possibilities we present, the more intermediate decisions the model must get right. It may choose the wrong tool, mix domains, or follow an irrelevant instruction. Reducing the decision space removes many opportunities for error and can improve the answer while lowering cost and latency.

The goal, therefore, should not be the largest possible context window, but the smallest one containing everything required. A request might first pass through a small model, then retrieve the appropriate information and reach a tool, service, or larger model only if necessary. In many cases, the large model would never need to become involved.

Separating decision-making from generation seems fundamental to me. Much of a business agent’s work is deciding what to do next: classifying, routing, validating a condition, or choosing a tool. A small model may be enough for these tasks while offering advantages in latency, cost, privacy, and local execution.

There is also no reason why all of a company’s AI should pass through one model. One can classify, another use tools, another work with documents, and another generate language. The model then ceases to be the application and becomes one component in a system where context, permissions, observability, and data matter as much as its capabilities.

This approach brings us to a less novel but perhaps more important subject: information governance. What is the source of truth—the ERP, the PIM, a spreadsheet, or SharePoint? Who maintains each fact? What happens when two systems disagree? Automation forces us to make explicit what a person once resolved through experience. To answer reliably, we must know where information comes from, who owns it, and whether it remains current.

The economic assessment should focus on the process, not only the price per token. We can measure how much time a query saves, how many errors it prevents, how many requests a small model resolves, how many are escalated, and what each route costs. The return emerges when we improve the complete process.

Always using the largest model, with long prompts and the entire tool catalog loaded into every call, may work, but it is unlikely to remain efficient as volume grows. Optimization is not just about paying less per token. It is about avoiding the use of context and compute where they add no value.

In business, the difference will probably not lie in who can access the most powerful model, but in how they design everything around it. Often, we do not need much smarter AI. We need an architecture that supplies better information, removes what is irrelevant, and uses the right model at the right time. If it can also do so with fewer tokens, lower latency, and less cost, it will be a better solution and much easier to justify from a business perspective.