Your AI Program Is Solving the Wrong Problem
One of the most common challenges organizations face today is why so many AI initiatives struggle to move beyond the pilot phase.
by Neerav Shah
As an Industry Advisor to the George Fox University MBA program, I work with faculty and leadership to help connect emerging industry trends with the skills future business leaders will need to succeed.
One of the most significant shifts impacting organizations today is the rapid adoption of artificial intelligence. While much of the conversation focuses on models, tools, and technology platforms, the real challenge is leadership; understanding how to apply AI to improve decision-making, create business value, and drive organizational change.
In support of George Fox University’s focus on AI leadership, I regularly share perspectives from my work with organizations across retail, healthcare, financial services, media, and technology. The following article explores one of the most common challenges I see organizations facing today: why so many AI initiatives struggle to move beyond the pilot phase.
Walk into almost any AI strategy meeting right now and within ten minutes someone will say it: “We are a Microsoft shop.” Or: “We have already rolled out Copilot.” Or the question will turn to which model has the best benchmarks, which vendor has the strongest roadmap, which one can handle the most data at once.
I have heard versions of this conversation in a retail head office trying to reduce inventory waste. I have heard it in an aerospace organization managing complex maintenance schedules. I have heard it in healthcare systems trying to improve patient triage. Different industries, different stakes, different teams — but the same opening question.
And in almost every case, the conversation about tools started before anyone had clearly defined which decisions they were trying to improve.
That is the problem. Not the technology. Not the vendor. Not the model. The sequencing.
“The goal is not to maximize what you spend on AI. The goal is to maximize the value of every decision your organization makes.”
The Infrastructure Trap
There is a seductive logic to the model-first approach. Saying “we are a Microsoft shop” or “we have standardized on Copilot” feels like a decision has been made. It feels like progress. Procurement is familiar. Vendor scorecards are manageable. Benchmarks are easy to present in a slide deck.
But what I have consistently seen is that these conversations, however well-intentioned, bypass the question that actually determines whether AI delivers value: which decisions in this business need to be better, and what does it cost us when they are not?
Selecting a model before defining the decisions it needs to improve is equivalent to a logistics company selecting an engine specification before it knows what it is transporting, how often, or how quickly it needs to arrive. The conversation starts with horsepower when it should start with the cargo.
Consider a retailer. Every day, that organization makes millions of decisions:
- What inventory should be reordered?
- Which products should be stocked in a remote store?
- Which promotions should be offered to which customers?
- How should labor be allocated across stores?
- Which products should be discounted before they spoil?
- How should customer service issues be resolved?
Not one of these decisions requires the same level of intelligence. Not one requires the same amount of context. And not one should necessarily use the same model. The answer to which AI to use depends entirely on which decision you are trying to improve — a question that never comes up when the room has already agreed on a vendor.
The Attention Span Problem
Every AI model has a limit to how much information it can absorb and work with at once.
Plain English
Think of it as attention span. Just like people, some models are optimized for quick interactions. Others can process vast amounts of information before reaching a conclusion. The mistake many organizations make is assuming that every decision requires the equivalent of a research analyst — when many decisions only require the equivalent of a knowledgeable store associate.
A customer trying to locate a product on the shop floor does not need a model capable of processing years of transaction history. A merchandising analyst evaluating supplier performance, weather patterns, inventory levels, regional demand, and promotional effectiveness very well might. A maintenance engineer in aerospace diagnosing a component fault needs something different again.
The question is never: “Which model has the largest attention span?” The question is: “Which decisions actually require it?”
In fact, poor context can be worse than limited context. A model overwhelmed with irrelevant information may arrive at a slower, more expensive, and sometimes less accurate answer than a model given only the information it needs. More context is not always better context. Relevant context is better context. The best AI systems do not show the model everything. They show the model the right things.
When Consumption Became a Badge of Honor
Here is something I did not expect to see as often as I have.
In organizations across the sectors I work with, I have watched developers and product leaders proudly announce that they have “run out of tokens.” It is said with the energy of someone who has just pulled an all-nighter before a big launch. The implication is clear: I am working hard. I am deep in the AI. I am one of the people who gets it.
In some of those organizations, those people are being recognized as AI leaders.
I want to be careful here, because the intent is often genuine. But running out of tokens is not a measure of contribution. It is a measure of consumption. And organizations that reward the two interchangeably will find it very difficult to ever demonstrate the business value of what they are building.
81%
of enterprises say AI ROI measurement is a top governance problem
Larridin State of Enterprise AI, 2025
14%
of CFOs report a clear, measurable impact from AI investments to date
RGP survey of 200 US finance chiefs, 2025
70%
of top AI enterprises will route decisions across multiple models by 2028
IDC FutureScape, 2026
No CFO walks into a board meeting and proudly announces that the company consumed ten times more electricity than last year. No supply chain executive boasts about moving inventory through three additional warehouses before it reaches a customer. Efficiency matters everywhere else in the business. There is no reason AI should be exempt.
Volume is not value. Consumption is not sophistication. Outcomes are what matter.
The Real Cost Question
This is where the conversation needs to shift — from technology to margin.
The Real Cost Question
Choosing the wrong model for the wrong decision type can increase operating costs by 10x, 50x, or even 100x at enterprise scale. A fraction of a penny per interaction sounds insignificant — until you are making millions of decisions every day. At that point, what appears to be a technology decision becomes a financial decision.
I have seen organizations become genuinely obsessed with model capability while paying almost no attention to model economics. In one case, a team had deployed their most powerful — and most expensive — model to handle a task that was repeated hundreds of thousands of times a day and required, in practice, about four lines of context to answer reliably. The model was doing it correctly. It was also costing a multiple of what it needed to.
Most businesses do not need the most powerful model for every task. They need the most cost-effective model that achieves the desired outcome. Deploying your most expensive model to every problem is the AI equivalent of dispatching a private jet every time someone needs to travel across town. Yes, it will work. But it is one of the least efficient ways to solve the problem.
The Decision Architecture Advantage
The most mature AI organizations I have worked with have stopped thinking about AI as a model selection problem. They think about it the way a logistics operator thinks about a fleet: different vehicles for different loads, different models for different decisions.
Simple, high-frequency decisions are routed to fast and inexpensive models. Real-time customer interactions are routed to low-latency models. Large analytical workloads—the kind a healthcare system might run across years of patient data, or an aerospace organization might run across thousands of maintenance records—are routed to models with the attention span to hold them. Complex reasoning tasks, where a wrong answer carries real consequences, are routed to premium models where the investment is justified.
The architecture makes the routing decision. Not the vendor. Not procurement. Not whichever model the development team happened to build a proof of concept with last quarter.
“Routing simple tasks to capable but inexpensive models and reserving premium models for decisions that truly require them is not cost-cutting. It is operational discipline. And increasingly, it is becoming a competitive advantage.”
What Leaders Need to Do Differently
The conversation that data and intelligence leaders need to be driving is not “Which model should we standardize on?” It is not “Are we a Microsoft shop or a Google shop?” And it is certainly not “How do we build a governance model around token consumption?”
It should be:
- Which decisions create the most value when improved?
- Which decisions are made most frequently?
- What is the cost of getting those decisions wrong?
- How much context is actually required to make a good call?
- How quickly does an answer need to arrive?
- What level of accuracy is necessary before value is realized?
Balance those variables across your decision portfolio and you have an AI strategy. Pick a model first — or a vendor, or a platform — and you have a procurement exercise dressed up as a strategy.
AI does not create value when a model generates a response. AI creates value when a better decision changes a business outcome. That is a distinction that sounds obvious when written down and is almost universally ignored in practice.
The organizations that will demonstrate meaningful AI value to their boards over the next several years will not be the ones with the most impressive vendor contracts, the highest token consumption, or the developers who most frequently hit their usage limits. They will be the ones that mapped their highest-cost, highest-frequency, highest-impact decisions and built a disciplined architecture around them.
That is what it means to treat AI as a decision strategy rather than a technology purchase. And in industries facing margin pressure, rising customer expectations, and increasing demands on already stretched teams, making the right decision at the right cost may be the most important competitive advantage available.
The organizations that understand this will stop asking: “Which model should we use?” And start asking: “Which decisions are we making better — and what is that actually worth to the business?”
Sources
Larridin State of Enterprise AI Report, 2025. Study of 350 senior Finance and IT leaders at companies with 1,000+ employees. businesswire.com
RGP CFO Survey, 2025. Survey of 200 US finance chiefs across technology, healthcare, financial services and retail sectors. cfo.com
IDC AI and Automation FutureScape, 2026. idc.com



