NetApp Insight 2024: Navigating the Shift Toward AI Inference and Sovereign Data Infrastructure

As the autumn conference season gathers momentum, the technology industry’s focus has shifted toward the practical implementation of the theoretical gains made in artificial intelligence over the past eighteen months. This week, the spotlight turned to NetApp’s annual Insight event, held in a surprisingly rainy Las Vegas. The atmosphere at the gathering reflected a pivotal moment for the enterprise storage giant, as it seeks to position itself at the heart of the "AI factory" while navigating a complex landscape of rising infrastructure costs and evolving safety concerns.

The event served as a platform for NetApp to articulate its vision for a future defined by smart storage, data readiness, and the industrialization of AI. However, the optimism of the keynote presentations was tempered by the reality of a challenging fiscal year and a shifting technological paradigm. As enterprises move past the initial hype of generative AI, the conversation is maturing, moving away from the raw processing power of GPUs toward the critical role of the data substrate that feeds these systems.

The Evolution of the AI Infrastructure Stack

A recurring theme throughout the conference was the changing nature of AI workloads. For much of the last two years, the industry dialogue has been dominated by the scarcity and necessity of high-end graphics processing units (GPUs). The focus was squarely on the training phase of large language models—a process that requires immense computational power. However, as Ross Kelly, ITPro’s news and analysis editor, observed live from the event, the focus is now shifting toward the next stage of the lifecycle: inference.

"I will say, anecdotally, I do think that storage is becoming the next big talking point when it comes to AI," Kelly noted during a dispatch from the conference. "We’ve gone from talking about GPUs; we’ve spoken about latency—latency is part of the story when it comes to storage—as we move away from training to inference, storage becomes a more important part of the AI infrastructure."

This shift is fundamental. While training a model is a resource-intensive, one-time or periodic event, inference is the ongoing process of the model providing answers and performing tasks in real-time. For an enterprise, inference happens every time a customer interacts with a chatbot or an automated system makes a decision. In this environment, the speed at which data can be retrieved and processed—the latency of the storage system—becomes the primary bottleneck. If the storage layer cannot keep up with the demands of the AI model, the entire "AI factory" grinds to a halt.

NetApp’s strategy appears to be leaning heavily into this transition. By rebranding the concept of the data center as an "AI factory," the company is emphasizing that data is the raw material that must be refined and delivered with precision. This requires a level of "data readiness" that many organizations currently lack. It is no longer sufficient to simply have large volumes of data; that data must be accessible, high-quality, and integrated into a storage architecture that can handle the high-throughput demands of modern AI models.

Economic Pressures and the Rise of Agentic AI