Bridging the AI Value Gap: Why Organizations Must Shift from Cost-Cutting to a ‘Return on Intelligence’

As the initial wave of artificial intelligence enthusiasm transitions into a period of rigorous corporate scrutiny, organizations are facing a sobering reality: while AI is undeniably boosting individual productivity, it has yet to move the needle on the corporate bottom line for the vast majority of enterprises. Industry analysts now suggest that the path to profitability requires a fundamental shift in how companies measure success, urging leaders to look beyond immediate cost savings toward a broader framework of "Return on Intelligence."

The disconnect between investment and impact has become a central theme in executive boardrooms. For nearly two years, organizations have poured billions into generative AI and large language models (LLMs), yet the financial returns remain elusive for most. Michael Chui, a senior fellow at the McKinsey Global Institute, notes that this phenomenon is not entirely unexpected, though the pressure to perform is mounting.

“There’s a delay between the development of technology, even the investment in the technology, and the value that an organization can capture from it,” Chui explained. This lag time is now testing the patience of stakeholders. According to Chui, the honeymoon period of experimental "sandboxes" is ending. “The CFOs are asking CIOs, investors are asking CEOs: ‘Where’s the ROI from this stuff, already?’”

The Productivity Paradox: Individual Gains vs. Organizational Profit

The current state of AI adoption is characterized by a striking paradox. Data from McKinsey’s “State of AI” survey, released in August, reveals that while 80% of respondents reported that AI had improved their individual productivity, that efficiency has not translated into corporate-wide financial performance. Only 37% of those surveyed could definitively say that AI’s impact was visible in their year-end profits—a figure that has remained stagnant since the previous year.

Even more telling is the rarity of "high-performing" AI organizations. Only 6% of respondents reported that AI delivered significant value, defined as accounting for at least 5% of their total operating profit. This suggests a massive "drop-off" between the value individual employees derive from using AI tools—such as drafting emails or summarizing meetings—and the value the organization as a whole captures.

Chui argues that the reason for this gap lies in the way AI is currently being deployed. Most companies are simply layering AI tools on top of existing, outdated processes. However, the most significant gains are expected to come from a total overhaul of the way work is done. According to McKinsey’s Technology Trends Outlook, the highest-performing companies are those that redesign their workflows from the ground up, creating environments where humans and AI agents work in a seamless, integrated fashion.

“Usually an end-to-end workflow involves multiple individuals, and completely redesigning that with the use of AI… is characteristic of high-performing companies,” Chui said. Without this structural change, AI remains a peripheral tool rather than a core driver of business value.

Controlling costs

As organizations attempt to scale their AI initiatives, they are running into a formidable obstacle: the unpredictable and often exorbitant cost of compute and tokens. In the world of generative AI, "tokens" represent the basic units of text or code processed by a model, and the costs associated with them can escalate rapidly as usage grows.

Chui emphasizes that realizing better returns is inextricably linked to managing these token costs and ensuring that the right model is being applied to the right task. Too often, there is a lack of transparency regarding which specific workloads are driving up the bill. Without a granular understanding of these expenses, companies risk spending more on the technology than the value it generates.

This concern is shared by IT leaders worldwide. Gareth Herschel, a vice president analyst at Gartner, speaking at the Gartner Data & Analytics Summit in Mumbai, noted that three out of five IT leaders are deeply worried about AI agents running up unexpected costs. In many cases, these fears have already been realized.

“Some organizations have already discovered that the cost of tokens for coding assistance is much higher than the cost of human software developers,” Herschel observed. This revelation challenges the common narrative that AI is a cheaper alternative to human labor. When an AI agent requires massive amounts of processing power to complete a task that a human could do efficiently, the economic argument for automation begins to crumble.

The financial risk is compounded as organizations move toward "multi-agent" systems, where various AI agents interact with one another to complete complex tasks. Robert Thanaraj, a senior director analyst at Gartner, compared the current state of unmonitored AI spending to a parent handing a teenager a credit card. “I’m sure you will learn a lot, but mostly from the bill,” he quipped.

To mitigate these risks, Thanaraj recommends that companies track costs meticulously during the prototyping phase. By calculating the cost of an individual agent per completed task, businesses can make more informed decisions about which models to use. This data-driven approach allows companies to evaluate whether they truly need a massive, expensive LLM or if a more affordable "small language model" or an open-weights model would suffice for the specific task at hand.

A wider lens for ROI

The difficulty in calculating AI’s return on investment stems from a variety of factors, including poor data quality, a failure to scale beyond initial pilots, and slow adoption among the workforce. However, analysts suggest that the problem may also lie in the definition of ROI itself.

While financial metrics are essential, Gartner analysts argue that focusing solely on immediate cash returns is too narrow a lens for a transformative technology like AI. In their Mumbai keynote, they noted that in 2025, the odds of any given AI initiative achieving a traditional financial ROI were only about one in five.

“ROI matters, but to achieve it, we must think of it not just as a financial metric, because value isn’t always just about money,” Thanaraj said. This has led to the emergence of a new concept: "Return on Intelligence."

Under this framework, companies are encouraged to tie AI projects to both financial and non-financial outcomes. This shift moves the emphasis from cost-cutting to value creation. Gareth Herschel explained that while revenue is a clear financial outcome, value can also manifest as improved "citizen experience" or customer satisfaction, brand loyalty, and long-term competitive positioning.

Despite the current challenges, there is evidence that the relationship between the finance and technology departments is maturing. Chui noted that CFOs and CIOs are beginning to develop new disciplines for tracking AI expenditures and returns more accurately. This cross-functional collaboration is seen as vital for moving AI from a speculative expense to a strategic asset.

The right foundation: context, infrastructure, governance

To bridge the gap between "amplified ignorance" and genuine business value, analysts agree that organizations must invest in a robust foundation consisting of data quality, technical infrastructure, and rigorous governance.

“Governance adds trust. Context adds meaning. Without strong foundations, AI may well stand for amplified ignorance,” Thanaraj warned. He pointed out that data quality remains one of the primary roadblocks to AI success. Because large language models are essentially sophisticated prediction engines, they rely heavily on the context provided to them. Without clear, high-quality data, these models are prone to "hallucinations"—generating confident but entirely incorrect outputs.

Herschel added that this "context layer" is something that cannot be purchased as a plug-and-play solution. It must be built internally to reflect the unique needs, proprietary data, and specific business logic of the organization.

The technical requirements are equally demanding. Jack Gold, principal analyst at J. Gold Associates, emphasized the need for a robust networking backbone to handle the massive data movement required by AI. As agent-to-agent interactions become commonplace and mission-critical, the infrastructure must be able to support a dramatic expansion in the number and distribution of these agents across remote devices and locations.

To manage this complexity, a growing number of organizations are implementing what is known as an AI "harness." This is a software layer designed to control, coordinate, and govern models, tools, and workflows. The importance of this layer is backed by data: a global KPMG survey released last month found that while 55% of all organizations have a formal AI harness layer, that number jumps to 86% among organizations that report having already established a clear ROI from their AI investments.

The presence of a harness layer suggests that success in AI is not just about having the best model, but about having the best system for managing those models. Organizations that can successfully combine clear accountability, coordinated governance, and reliable value measurement will be the ones best positioned to turn broad AI adoption into sustained, long-term performance.

As the industry moves into 2025, the focus is shifting away from the novelty of what AI can do and toward the practical reality of what it should do for the enterprise. For those who can navigate the complexities of cost, context, and workflow redesign, the promise of the "Return on Intelligence" remains within reach.