Scaling the Frontier of Agentic Workflows: Michael Levan on the Operational Realities of AI Agents

This level of intelligent traffic management is crucial for running AI at scale. It prevents the over-provisioning of expensive resources and ensures that the most capable models are reserved for the tasks that actually require their advanced reasoning capabilities. For platform engineers, this represents a new frontier where networking logic is driven by natural language processing and embeddings rather than static rules.

The Necessity of Sandboxing and Execution Security

Perhaps the most pressing security concern discussed by Ned, Kyler, and Levan was the issue of execution. If an AI agent is tasked with analyzing a financial spreadsheet, it may decide to write and execute a Python script to perform the calculation. If that execution happens on a local server or within a sensitive environment, the risks are immense.

Sandboxing emerged as a non-negotiable best practice for agentic workloads. Levan highlighted the importance of creating isolated, ephemeral environments where agents can execute code without posing a threat to the broader system. These "digital playrooms" must be strictly governed, with limited access to the network and file systems.

The challenge, as the participants noted, is balancing security with performance. A sandbox that takes too long to spin up can cripple the responsiveness of an AI agent. Therefore, the infrastructure must support high-performance, low-latency containerization or micro-VM technologies that can provide isolation at the speed of the agent’s thought process.

Robust Observability in a Non-Deterministic World

Traditional observability focuses on metrics like CPU usage, memory, and request latency. While these remain important, the group agreed that AI agents require a new category of observability. Because agents are non-deterministic, understanding why an agent made a particular decision is just as important as knowing that it completed its task.

Levan and the hosts discussed the need for "traceability" in agentic loops. This involves capturing the full chain of thought, the tools used, and the intermediate outputs generated by the agent. Without this level of granular detail, debugging an agent that has "gone off the rails" becomes nearly impossible.

Observability also plays a key role in cost management. Since agents can enter loops where they repeatedly call an LLM to refine a task, organizations need real-time visibility into token consumption and "recursive depth." The discussion underscored that robust observability practices are the only way to ensure that autonomous agents remain both effective and economically viable.

AdSpot Sponsor: Gartner

The complexities discussed by Levan, Ned, and Kyler are exactly the types of challenges that the upcoming Gartner IT Infrastructure Operations and Cloud Strategies Conference aims to solve. Scheduled to take place from December 8th through the 10th in Las Vegas, the conference is a premier gathering for heads of infrastructure, cloud operations professionals, platform engineers, and security leaders.