An application can use JavaScript in one component and Java in another. Those choices can reflect different requirements, existing systems, team capabilities, and operating constraints. We do not need to settle the question of which language is universally best before designing the application.
I think we should approach AI models with the same discipline.
In my experience, one vendor and one model are not a panacea. Different parts of a product can benefit from different capabilities and cost structures. Models and agents should be evaluated in relation to the work they need to perform.
That changes the conversation for technology and product leaders. A model comparison can be useful, but a ranking alone cannot determine what belongs in your system.
Which model performs this task well enough, within the constraints of this product, at an acceptable total cost?
That is a question we can test. It also leaves room for the answer to change.
One product can contain several different kinds of work
Consider a hypothetical application that helps a business process incoming requests.
One task is to identify the request type. Another is to interpret an attached document. A third is to work through an unusual case involving several conflicting pieces of information. The application may then prepare an action in another system.
These tasks do not necessarily need the same model, the same amount of computation, or the same level of autonomy.
A fast model might be sufficient for a well-defined classification task. Document interpretation might require capabilities that the first model lacks. A difficult case may justify a more capable model or a person. An action requires appropriate tools and permissions in addition to a good answer.
This is an architectural possibility, not a rule that every application should contain multiple models. Testing may show that one model serves all the tasks adequately. The point is to make the choice at the level where the requirements differ.
Possible routes within a product. Each route needs evidence that it meets the task's requirements; a request need not pass through every model.
Models, agents and vendors are different decisions
The terms are sometimes used interchangeably, but they describe different parts of the system.
A model generates, interprets, or reasons over information. Its suitability depends on the task and the context it receives.
An agent combines model capability with instructions, tools, state, and a process for carrying out work. Changing the model inside an agent can change how it selects tools or responds to an unexpected result.
A vendor provides access under particular operating and commercial arrangements. Choosing one involves service reliability, integration, data requirements, support, and cost as well as model capability.
Anthropic's engineering guidance makes a related distinction between predefined workflows and agents that dynamically determine their own steps and tool use. It also recommends using simple approaches when they meet the need. These are useful design principles even when the final application uses other providers. See “Building effective agents.”
Selecting a different model is therefore not always equivalent to selecting a different agent. Nor does adding another vendor automatically provide a new capability. Leaders need to understand which part of the system they are changing and what benefit they expect.
These choices interact. A change to one component can alter the behavior and economics of the complete system.
Think in terms of an AI compute portfolio
I find it useful to think of models as a portfolio of available capabilities. The objective is to allocate work sensibly across them.
We already make similar choices in software architecture. We select storage, databases, and compute resources according to the work they need to support. Using the most expensive option everywhere rarely replaces the need to understand requirements.
For AI, that portfolio might initially contain one general-purpose model. A second could be added when a repeated task has a different cost or quality requirement. Another might address a specific input type or operating constraint.
The portfolio should remain small enough to understand. Every addition creates something to integrate, evaluate, monitor, and maintain.
Routing research provides evidence that selective model use can be useful. RouteLLM investigates choosing between stronger and weaker models according to the request, with the aim of balancing cost and response quality. Its benchmark results support the possibility of savings; they do not guarantee the same outcome for a particular production workload. Read the paper.
For a product team, the important next step is to test the principle on its own tasks, rather than adopt a routing system simply because the architecture looks sophisticated.
Routing can begin with a clear rule
The first version of a routing policy does not need to be another intelligent agent.
If requests arrive in known categories, a team can decide which model handles each category. If a task requires a particular input capability, that requirement can determine the route. If the information cannot be sent to a particular provider, that provider should be excluded before cost is considered.
More dynamic routing may become useful when tasks vary in ways that cannot be captured by simple rules. That flexibility introduces another component whose decisions need evaluation.
A router that regularly sends difficult cases to an inadequate model can erase the savings it was intended to produce. Sending everything to the most capable model may avoid that routing mistake while creating unnecessary cost or delay elsewhere.
An escalation path helps, provided there is a meaningful signal for using it. A model's statement that it is confident should not be treated as sufficient evidence that its answer is correct. Checks might include required information, consistency with a source, task-specific validation, or human review.
The system also needs a stopping point. Repeatedly passing a failed task between models can consume resources without improving the result.
Measure the cost of successful work
A quoted usage price is an input to the decision. The product experiences the cost of the complete interaction.
That may include context sent to the model, generated output, repeated attempts, tool calls, supporting services, and review. Response time also matters: a cheaper result that arrives too late may be unsuitable for the customer experience.
I would compare the total cost of completing representative tasks at an acceptable quality level. This makes the tradeoff more meaningful than comparing nominal prices in isolation.
For one task, a cheaper model may be entirely sufficient. For another, a more expensive model may produce a usable result with less correction. The answer can differ between the same two models depending on the work.
Commercial arrangements also need to be assessed against the actual pattern of use. Predictable volume and occasional bursts may produce different priorities. The team should examine its likely usage, commitments, and operational overhead rather than assume that the lowest advertised rate produces the lowest bill.
Interchangeability has to be earned
The JavaScript and Java analogy also reminds us that combining technologies requires design.
A common interface can reduce the effort of connecting a new model. It cannot guarantee that the model will interpret instructions in the same way, return equivalent outputs, or use tools with the same reliability.
I would keep the business task clearly defined: what information goes in, what a successful result contains, which actions are permitted, and what happens when the task cannot be completed.
Then I would maintain representative examples, including edge cases and known failures. Candidate models should be evaluated against those examples and against the product behavior that matters to users.
Some instructions or integrations may need to remain specific to a model or provider. Pretending otherwise can hide differences that matter. A useful abstraction keeps those differences manageable and visible.
This is particularly important for agents that change external systems. A replacement model may produce a plausible answer yet choose a different action. The evaluation has to cover the action and its consequences, not merely the wording of the response.
Switching is an operating capability supported by evaluation and controlled release. A compatible connection is only the starting point.
Another vendor should solve a real problem
There are good reasons to consider an additional provider: a capability gap, a better cost structure for particular work, or a tested alternative when the primary service is unavailable.
There are also costs. Teams must manage another integration, evaluate data handling, monitor another dependency, and keep the fallback working as the application changes.
A fallback is useful only if it is allowed to receive the relevant information and can meet an acceptable quality threshold. Quietly switching to a provider with different constraints could turn a service interruption into a larger problem.
Sometimes a limited response or a queued task is a better fallback than an untested substitute. That should be a deliberate product decision.
For a small team, a single provider may remain the most sensible starting point. The objective is to keep future choices possible where they matter and to add complexity when a demonstrated benefit justifies it.
Make model selection an ongoing product discipline
Technology leaders can establish the interfaces, evaluation process, and operational controls. Product leaders need to define what a successful outcome looks like, how much delay is acceptable, and which failures have meaningful consequences for customers.
Together, they can periodically reassess whether the current choices still fit. A new model may improve one task while offering little benefit elsewhere. Changes in usage can alter the economics. A provider change may require retesting assumptions that once held.
I believe the strongest approach is to treat models, agents, and vendors as choices within a larger design. The product remains the point of reference.
The useful question is how well the system serves its customers and business. Choosing its components with that purpose in mind gives us a much better basis for progress than expecting one model to be the answer to everything.