The best AI agents are simpler than you think

The Future of Agentic Commerce

Introduction to Sierra and Agentic Commerce

• Zack Reno Wedeen discusses the potential of agentic commerce surpassing traditional e-commerce, highlighting commission-based sales through Sierra agents.

• Sierra is positioned as a platform for customer experience agents, primarily serving Fortune 20 companies.

Unique Architecture of Sierra Agents

• Wedeen explains how Sierra's architecture differs from standard agent harnesses by enabling parallel processing of thinking, listening, and talking.

• Conversations with Sierra involve multiple models working simultaneously, enhancing responsiveness compared to typical LLM interactions.

Payment Infrastructure in Sierra

• A separate infrastructure layer for payments ensures that sensitive information does not interact with external LLM providers due to PCI certification concerns.

Expanding Beyond Customer Support

Vision for Full Engagement Platform

• Initially focused on customer service solutions, Sierra aims to be an all-encompassing engagement platform addressing various customer interaction points.

• Examples include flight booking processes where agents assist throughout the entire journey from browsing to post-flight issues.

Diverse Use Cases and Commission Model

• Different types of interactions (sales, service, loyalty) are facilitated by Sierra agents across the customer lifecycle.

• The outcome-based pricing model allows agents to earn commissions on sales, expanding their role beyond traditional service expectations.

Building on the Sierra Platform

Extensibility and Customization

• The platform is designed for extensibility; users can customize agents significantly while maintaining a consistent starting point.

Structure of the Building Process

• Users navigate through three main sections: analyze, build, and release. Each section serves distinct functions in developing effective agents.

Analyzing and Iterating on Agents

Analysis Before and After Release

• Analysis can occur both before building an agent (using transcripts or SOP documents as resources), and after deployment for ongoing optimization.

Roles in Improvement Processes

• Primarily operations personnel (customer experience managers), along with engineering teams, engage in analyzing performance metrics and iterating improvements.

No-Code Agent Building Experience

Overview of No-Code Capabilities

• The no-code experience allows users without programming skills to create complex agent behaviors using natural language instructions rather than raw code.

Ghostwriter's Role in Development

• Ghostwriter assists users by generating journeys based on user prompts without writing traditional code but instead creating structured workflows directly within the system.

Balancing Abstractions in Model Training

Challenges with Model Understanding

• There’s a constant tension between aligning model training data with ideal abstractions versus adapting user needs into formats that models understand effectively.

Evolution of Agent SDK

Changes Over Time

• The evolution from flow-based structures towards more sophisticated reasoning capabilities reflects advancements in both models used and orchestration platforms available.

Inter-Agent Communication

Protocol Choices

• Most common communication between agents occurs via API calls; however, support exists for MCP protocols allowing flexibility depending on enterprise needs.

Future Outlook: Agentic Commerce vs E-Commerce

Predictions about User Behavior

• Wedeen predicts that personal assistants will dominate future commerce interactions over traditional website navigation as brands prepare for this shift toward agent-driven transactions.

Insights on AI Agents and Their Applications

The Value of Human Interaction

• The speaker emphasizes that human attention is more valuable than automated systems generating tokens without engagement, suggesting a future where human interaction remains crucial.

Presenting Information Effectively

• Importance is placed on how agents present themselves, making it easy for users to understand available products and preferences, which remains relevant in commercial contexts.

API Interactions with Agents

• Discussion about companies like Sentry providing APIs for direct agent interactions highlights the need for brands to care about platform usage and presentation.

Partnering for Better Solutions

• The financial stakes involved in AI solutions lead companies to partner with established firms (like Sierra) to ensure optimal outcomes, even if they can achieve 90% effectiveness independently.

Early Adoption of Payment Systems

• Both speakers agree that they are still early in adopting payment systems through AI agents, indicating a gap between potential and current usage levels.

Subscription Management Services

• There’s a growing interest in apps that manage subscriptions effectively; the speaker feels closer to using Codex for this task compared to existing applications due to its manual nature.

Speculative Execution in Knowledge Retrieval

• The concept of running multiple processes simultaneously is discussed, particularly regarding knowledge retrieval where answers are prepared before confirming their necessity.

Modular Architecture Advantages

• A modular architecture allows flexibility across various languages and use cases by utilizing different models concurrently for tasks like transcription based on specific needs.

In-House Model Development Rationale

• The decision to develop in-house models stems from the need to push boundaries when existing models limit customer service capabilities or industry-specific requirements.

Balancing Frontier Models with Custom Solutions

• While large training runs produce advanced models (e.g., GPT), the focus should be on understanding customer processes deeply rather than solely relying on these frontier technologies.

Context Engineering as Key Strategy

• Effective context engineering involves providing agents with necessary information while avoiding overload; this balance improves performance and reduces errors during interactions.

This markdown file captures key insights from the transcript while maintaining clarity and organization. Each bullet point links directly back to its corresponding timestamp for easy reference.

Voice Experience Development

Introduction to Voice Project

• The speaker describes their experience working on the voice project at Sierra, highlighting it as one of the most enjoyable projects in their career.

• They joined Sierra as an agent PM focused on building agents for customers, starting with SiriusXM, a major in-car streaming radio service.

Key Considerations for Voice Experiences

• The development process involved thinking from first principles about what constitutes a great voice experience and how it differs from chat interactions.

• Latency is crucial; developers must be mindful of parallelism and progress indicators during conversations.

Naturalism and Multilingualism Challenges

• Naturalism combines various factors that affect how human-like the voice sounds, including both the agent's script and voice quality.

• Multilingual support poses challenges due to varying transcription accuracy across languages, necessitating ensemble approaches to improve performance.

Advancements in Voice Technology

Real-Time Voice Models

• Recent advancements include production agents using real-time voice-to-voice models, enhancing interaction fluidity while still requiring transcripts for API calls.

Balancing Speaking and Listening

• A significant design breakthrough was achieving a balance between deciding when to speak and what to say, allowing simultaneous listening and thinking.

Modularity in Voice Solutions

Importance of Modularity

• No single provider excels at all aspects of voice technology; modularity allows flexibility in choosing different providers based on specific needs.

Future of Native Models

• While native voice-to-voice models are improving, they currently lack reliability across multiple languages and remain more expensive than traditional methods.

Memory Integration in AI Agents

Significance of Memory

• Memory plays a critical role within Sierra’s platform by enabling agents to remember past interactions with users for improved service continuity.

Implementation Strategies

• Memory can be stored implicitly or explicitly during conversations. This includes remembering user preferences or previous issues encountered.

Challenges with Memory Systems

Trust Issues with Memory Usage

• Implementing memory systems requires trust regarding authentication since sensitive information may be involved.

Evaluation Processes for AI Agents

Internal vs. Customer Evaluations

• Evaluation processes differ internally versus customer-facing applications due to complexities like background noise or adversarial users affecting conversation quality.

Continual Learning Capabilities

Current State of Continual Learning

• The platform supports automatic issue detection but maintains human oversight before implementing changes suggested by AI agents.

This structured markdown file captures key insights from the transcript while providing timestamps for easy reference. Each section focuses on distinct themes discussed throughout the conversation.

Understanding Multimodal Experiences in Different Industries

The Importance of User Experience

• In industries like airlines, there is a growing demand for multimodal experiences that enhance reservation retrieval and input processes.

• Retail benefits from polished user interfaces (UI) focused on product discovery and recommendations, which significantly impact customer engagement.

Differentiation Through Customer Understanding

• Sierra stands out by deeply understanding specific customer needs, particularly in creating exceptional retail discovery experiences.

• Success in vertical companies stems from comprehending the unique characteristics of each industry, influencing their operational strategies.

Outcome-Based Pricing: A Game Changer?

Aligning Incentives with Customers

• Outcome-based pricing aligns the interests of Sierra and its customers, enhancing collaboration and resource allocation.

• Delivering significant outcomes allows both parties to benefit financially, fostering a unified direction in decision-making.

Future Trends in Pricing Models

• Companies engaged in high-value activities will likely adopt outcome-based pricing as a standard practice.

• Simple tasks may not command high premiums; however, substantial sales outcomes justify higher costs for businesses.

Navigating Complex Customer Interactions

Varied Outcomes Across Services

• The value of interactions varies significantly between customer support and sales; pricing structures depend on the complexity of tasks involved.

• Some interactions yield high-value outcomes while others are more commoditized, leading to differentiated pricing strategies within the same customer base.

Trust and Long-Term Relationships

• Maintaining aligned incentives fosters trust over time; negotiating every detail can be counterproductive to building strong partnerships.

The Role of Agency in Engineering at Sierra

Characteristics of Successful Team Members

• Employees who thrive at Sierra exhibit deep customer intuition and agency—qualities essential for navigating complex enterprise environments.

• Balancing craftsmanship with an understanding of consumer-grade products is crucial for success within the company’s framework.

Evolving Skills Required for Agent Builders

• As coding agents become prevalent, skills such as product judgment and communication are increasingly vital to maintain efficiency without bottlenecks.

Interviewing for Agency: A New Approach

Innovative Interview Techniques

• The AI-native interview process involves candidates building a product end-to-end within hours, showcasing their sense of control and initiative.

Assessing Candidate Potential

• This method reveals how candidates perceive their roles' boundaries and whether they can identify opportunities beyond traditional scopes.