You're Building AI Agents on Layers That Won't Exist in 18 Months. (What this Means for You)

Understanding the New Infrastructure Stack for AI Agents

Introduction to the New Infrastructure Stack

• A new infrastructure stack for AI is emerging, attracting significant investment but often overlooked by many.

• The video aims to clarify this new category of infrastructure for agents and help viewers understand its components and implications.

Historical Context of Infrastructure Shifts

• Previous shifts in computing infrastructure include the transition from on-premise servers to cloud services (2006-2010), exemplified by AWS.

• Another shift occurred between 2012 and 2016, moving from monolithic applications to microservices architecture with APIs.

Current Shift: Human Tools to Agent Primitives

• The current transition involves moving from human-first tools to agent-first primitives, which is foundational like the move to cloud computing.

• The new customer for infrastructure will be agents, similar to how enterprises became customers of compute resources in the past.

Challenges in Distinguishing Real Innovations

• There is confusion in distinguishing genuine innovations from hype due to the novelty and size of startups involved.

• A reliable analogy for current developments is system calls; agents require defined interfaces for identity, compute, memory, communication, and payments.

Layer One: Compute and Sandboxing

• The first layer focuses on compute and sandboxing, crucial for providing a safe environment for code execution.

• E2B has raised $32 million using firecracker microVM technology aimed at offering dedicated kernel sessions per agent.

• Other players include Daytona with Docker containers optimized for speed and Modal targeting GPU-heavy workloads.

• Philosophical differences exist regarding whether agents should be ephemeral or persistent; both approaches have their merits depending on workload needs.

• This layer's maturity indicates multiple options available as startups continue developing solutions tailored for virtual environments.

Agent Identity and Communication in the Digital Space

The Role of Identity and Communication for Agents

• Layer two focuses on identity and communication, emphasizing the need for agents to exist as entities on the internet capable of sending and receiving messages.

• Agent Mail recently raised $6 million to create programmatic email inboxes for agents, highlighting email's role as a fundamental identity layer rather than just a communication tool.

• Email serves as a universal key for online services; however, its effectiveness is questioned due to inherent issues like spam prevention measures that hinder agent functionality.

Challenges with Current Email Protocols

• The reliance on email may be more about human convenience than suitability for agents, raising concerns about its appropriateness as an identity protocol.

• Issues such as brittle threading and poor signal-to-noise ratios complicate effective communication for agents, indicating a need for better protocols that do not require mimicking human behavior.

Emerging Solutions in Agent Identity

• Various teams are exploring alternatives like on-chain identities and dedicated communication standards; however, no clear winner has emerged in this evolving space.

• While email remains resilient over time, betting on it as the sole solution for agent identity may be pragmatic rather than architecturally sound.

Memory and Statefulness in Agents

Importance of Memory for Agents

• Agents require memory capabilities that extend beyond single sessions to maintain context across multiple tasks over time.

• Mem0ero leads this sector with significant funding and adoption metrics, recognized by AWS as a primary memory provider.

Innovative Approaches to Memory Management

• Mem0ero emphasizes active curation of memory—storing relevant information while discarding outdated details—to enhance contextual understanding during interactions.

• Their architecture combines various data storage formats (network graph, vector database, key-value store), allowing them to manage memory effectively rather than treating it merely as an add-on feature.

Competitive Landscape in Memory Solutions

• As major AI labs invest heavily in integrating long-term memory into their models (e.g., OpenAI's ChatGPT), standalone memory companies face potential risks if they cannot compete with these integrated solutions.

The Future of Memory Ownership

Portability vs. Convenience Debate

• A critical discussion revolves around whether users should own their memories or rely on hyperscalers' convenient solutions; this could shape market preferences significantly.

Uncertain Market Dynamics

• The outcome remains uncertain: will users prioritize independent memory solutions or gravitate towards the ease offered by larger tech companies?

Tools and Integration Needs for Agents

Expanding Tool Integration Capabilities

• As agents increasingly interact with various tools (e.g., Slack, Jira), there is rapid growth in integration layers designed to address immediate operational challenges faced by these digital entities.

Integration Challenges in Enterprise Environments

The Role of Managed Integration Layers

• Discusses the necessity of integrations and basic primitives like Unix or Python for enterprise-level work, emphasizing that tools facilitating these integrations are crucial.

• Introduces Compose, which has received $29 million in funding to provide a managed integration layer for agents, simplifying authentication and offering pre-built connectors to numerous solutions.

The Complexity of Agent Integration

• Highlights the "NSM integration nightmare," where each agent builder must manage various aspects like credentials and API changes independently, creating an unsustainable combinatorial problem.

• Argues that as long as tool ecosystems remain fragmented, there will be a durable need for integration layers. However, standardization could diminish this value if universal MCP (Managed Control Plane) systems become prevalent.

Trust Layer Development

• Introduces the concept of a trust layer for agents to securely acquire services and make payments, with Stripe Projects being a recent development in this area.

• Explains how agents can provision databases and upgrade hosting tiers using Stripe while ensuring payment details remain secure within Stripe's vault.

Future Growth Areas in Agent Transactions

• Predicts growth around agent-to-agent payments, metered billing aligned with compute patterns, and dynamic budget allocation based on approval levels.

• Emphasizes that future infrastructure will prioritize agent legibility over human readability while maintaining observability for humans.

Orchestration Opportunities in Multi-Agent Systems

Current Gaps in Agent Coordination

• Identifies orchestration as a significant opportunity due to the need for reliable multi-agent collaboration at scale with features like fallback handling and audit trails.

• Notes that existing tools focus on framework-level capabilities rather than comprehensive infrastructure needed for running multiple agents reliably across enterprise systems.

Essential Infrastructure Needs

• Stresses the importance of developing layers that enable composable capabilities among agents while ensuring reliability and parallel processing.

Agent Infrastructure and Orchestration Challenges

Key Components of Agent Infrastructure

• The need for a life cycle layer for agents that manages creation, assignment, health checking, scaling, and termination as a service.

• Importance of merge and coordination infrastructure designed for parallel agent work, including conflict detection and resolution protocols.

• Necessity for supervision hierarchies with meta agents to monitor and evaluate other agents without requiring custom coding.

• Financial observability across multiple workflows is essential to track spending, outcome quality, and cost per successful task—akin to FinOps for agents.

• Standard failure patterns and recovery protocols are crucial to ensure consistent responses when an agent's tool call fails.

The Future of Agent Orchestration

• The orchestration layer is seen as the next frontier in infrastructure development, similar to what Kubernetes achieved with container orchestration.

• Current reliability issues arise from dependencies on multiple primitives; end-to-end reliability diminishes as more components are added.

• Transitional lock-in poses risks when using temporary solutions (shims), which may complicate future migrations once native protocols emerge.

Risks Associated with Agent Development

• Strategic choices regarding technology adoption are critical; each decision can lead to either standardization or increased migration costs later on.

• Anticipation of "agent sprawl," where excessive reliance on agents leads to operational chaos without proper observability or orchestration layers.

Essential Skills for Builders in the Agent Space

• Context engineering is vital; the input provided to agents significantly influences their output and effectiveness.

• Emphasis on evaluation-driven development allows agents to autonomously achieve results while minimizing human bottlenecks in code review processes.

• Stack literacy becomes increasingly important; understanding which layer offers competitive advantages is crucial for success in building effective agent systems.

Conclusion: Navigating the Agent Landscape

• Builders must maintain awareness of how changes within the stack affect their business strategies; lack of stack literacy can lead to significant challenges.

Understanding the Importance of Agent Stack Literacy

The Role of Agents in Business Outcomes

• Agents significantly influence business outcomes and leverage, making it crucial to understand the components of the technology stack that support them.

• A comprehensive understanding is necessary regarding which parts of the stack are effective, which are custom-built, and what dependencies exist within the system.

• Without this detailed knowledge, relying on agents becomes a gamble rather than a strategic decision.

The Need for Better Stack Literacy

• Long-term success requires improved literacy around the agent stack; merely using buzzwords without comprehension can lead to negative consequences.

• Many individuals in engineering roles may lack a fundamental understanding of how the agent stack operates, leading to potential challenges in their work.

• Sharing educational resources about the agent stack is essential to mitigate confusion and enhance overall effectiveness within teams.