Building Microsoft AI Agents: Which Tool Should You Use?

Understanding Microsoft AI Agents

Overview of AI Agent Building Options

• Microsoft offers various methods to build AI agents, which can be overwhelming. The speaker aims to clarify the options available, including Co-pilot Studio, Agent Builder, Azure, and code-first solutions.

• There are two primary types of agents on the Microsoft platform: those that extend Microsoft 365 Co-pilot and standalone custom agents.

Types of Agents

Extending Microsoft 365 Co-pilot

• The first type is an extension of Microsoft 365 Co-pilot, a paid service at $30 per user per month. It connects to all graph data and integrates with the Co-pilot chat interface.

• This agent type is designed for internal use within organizations and requires authentication.

Standalone Custom Agents

• The second type is a completely standalone agent that may not connect to any Microsoft 365 or tenant data. This could involve external systems or custom AI solutions.

Skills and Tools for Building Agents

Skill Levels of Makers

• Consideration must be given to the skills of the maker or team involved in building these agents. Options range from natural language processing using Agent Builder for everyday users to low-code solutions via Co-pilot Studio.

• For advanced developers, there are code-first tools that allow deep customization of large language models and user experiences.

Anatomy of an AI Agent

Key Components

• Understanding what you need to control when building an agent includes instructions (behavior), knowledge sources (documents/databases), action triggers (workflows), and decision-making processes (orchestrator).

• Foundational models refer to the underlying language models used by these agents; various providers offer different options.

Control vs. Ease of Use

• The level of control over these components varies based on the chosen approach—code-first provides full control while extending Microsoft 365 limits it but simplifies development time.

Extending Microsoft 365 Co-pilot: Use Cases

Benefits and Limitations

• Extending Microsoft 365 Co-pilot allows users access to foundational models like OpenAI's GPT without needing extensive technical knowledge or understanding orchestration details.

• These extensions are referred to as declarative agents in coding terms; however, business users may simply view them as enhancements within their existing tools.

Conclusion on Choosing Agent Types

Decision Factors

How to Create and Use Microsoft 365 Co-Pilot Agents

Overview of Microsoft 365 Co-Pilot Capabilities

• The chat UI includes suggested prompts, facilitating user interaction. A demo will illustrate this functionality.

• Users with a Microsoft 365 co-pilot paid license can easily connect to business data, including SharePoint and OneDrive.

• Even without a license, users can access the Microsoft 365 environment for basic functionalities.

Custom Solutions vs. Internal Use Cases

• For customer-facing solutions or custom channels, users should refer to building custom agents instead of using the standard co-pilot features.

• The internal use case requires logging in with a Microsoft work account; external extensions are not supported within the co-pilot framework.

Control and Customization Options

• Users can customize agent instructions (e.g., tone of voice), connect to organizational knowledge bases like SharePoint, and set up workflows.

• While some autonomy options may be available in future updates, foundational models and user experience remain fixed within the chat interface.

Building Agents as an End User

• The Co-Pilot Studio Agent Builder allows end users to create agents using natural language without advanced skills.

• Users can access pre-built agents that assist with common tasks directly from their Microsoft 365 co-pilot chat experience.

Examples of Pre-Built Agents

• Various pre-built agents are available for learning assistance or career coaching, which provide structured guidance based on user queries.

• These everyday co-pilots come equipped with predefined instructions similar to creating custom GPT models but are tailored for specific functions.

Creating Custom Agents for Specific Needs

• Users can create personalized agents focused on particular projects or topics by simply typing requests in natural language.

Creating an Agent: Travel and Expense Assistant

Overview of Agent Creation

• The process begins with creating an agent named "Travel and Expense Assistant," which suggests various prompts for user interaction.

• Users can refine the agent's behavior through configuration options, including AI-generated instructions based on a brief description provided by the user.

Connecting Knowledge Sources

• Users have the option to connect the agent to specific knowledge sources, such as an HR Library or individual documents, enhancing its functionality.

• Business users can create agents tailored for specific use cases directly within SharePoint, allowing for seamless integration into their workflow.

Extending Agent Functionality in SharePoint

Creating Agents in SharePoint

• Agents can be created alongside existing documents in SharePoint, enabling users to ask questions directly related to their content.

• The creation process is similar to previous methods but starts from a knowledge source rather than a chat interface.

Transitioning to Co-Pilot Studio

• The next level involves transitioning from business users to low-code makers who can further enhance agent capabilities using Co-Pilot Studio.

• Users can edit and extend agents within SharePoint, adding advanced customization options that are not yet fully available.

Integrating Microsoft 365 Co-Pilot with Other Systems

Extending Capabilities with Co-Pilot Studio

• Agents created in earlier steps can be extended using Co-Pilot Studio, allowing for more sophisticated functionalities and actions against other systems.

• This extensibility means that initial creations are not limited; they can evolve into more complex agents over time.

Code First Approach vs. Declarative Agents

Understanding Code First Programming

• A code-first approach offers greater control and sophistication compared to declarative agents created through drag-and-drop interfaces.

• Tools like the Teams Toolkit allow users—even those without coding experience—to define agent behaviors and capabilities effectively.

Conclusion on Agent Development

Building Custom Agents with Co-Pilot Studio

Introduction to Custom Agents

• The speaker discusses the limitations of using a large language model interface, emphasizing the desire for more control and customization in building agents.

• Custom agents allow users to publish on external channels beyond Microsoft 365, providing greater control over knowledge, applications, workflows, automation, and orchestration.

Tools and Development Approaches

• The development process is described as a stack from low code to code-first approaches; moving up the stack increases user control.

• Co-Pilot Studio offers a low-code experience suitable for citizen developers, allowing users to create agents without extensive coding knowledge.

Creating an Agent in Co-Pilot Studio

• Users can start with templates in Co-Pilot Studio for various applications like IT help desk queries or website-based Q&A.

• An example of an IT Help Desk agent is provided; it connects to non-Microsoft systems (e.g., ServiceNow), allowing users to input knowledge and actions.

Advanced Features of Agents

• A "Healthy Living Assistant" agent example demonstrates connecting to external websites and performing actions like fetching weather forecasts.

• Users can influence conversation orchestration by defining manual topics (e.g., emergency contact numbers).

Publishing and Autonomous Agents

• Co-Pilot Studio supports publishing across approximately 15 different internal and external channels.

• Autonomous agents can operate without human intervention; an example includes a returns agent that processes customer requests via email based on predefined policies.

Extending Capabilities with Azure AI Foundry

• For those seeking more control over models or data sets, integration with Azure AI Foundry allows for advanced customization beyond what Co-Pilot Studio offers.

Understanding Co-Pilot Studio and Azure AI Integration

Integrating Knowledge Sources into Co-Pilot Studio

• A scientist in the team has developed connections to extensive knowledge sources within the organization, implementing sophisticated indexing that can be integrated into Co-Pilot Studio.

• Users can incorporate various language models beyond the standard offerings in Co-Pilot Studio, including those from different suppliers or smaller models tailored for specific needs.

• By integrating Azure AI search capabilities, organizations can leverage existing data sets indexed by data scientists, enhancing control over their Co-Pilot Studio agents without managing the underlying infrastructure.

Custom Data Models and Extensibility

• Users have the option to integrate custom data models (e.g., a finance model developed internally), allowing for tailored responses based on specific organizational needs.

• The extensibility of Co-Pilot Studio means users are not limited; they can start with basic functionalities and progressively incorporate more complex customizations as needed.

Choosing Between Approaches: Low-Code vs. Code-First Tools

• For complete control over user experience and data models, developers may opt for code-first tools available in Azure AI Foundry, which require setting up personal infrastructure but offer greater customization options.

• Both approaches—Co-Pilot Studio for rapid deployment and code-first tools for deeper customization—can coexist; understanding their strengths is crucial for effective agent building.

Cost Considerations and Business Justification

• Using Co-Pilot Studio allows quick setup without worrying about infrastructure costs or maintenance; it operates on an all-in-one pricing structure based on message types.

• In contrast, utilizing Azure AI resources involves separate pricing for components like indexing services and language models, necessitating careful cost modeling to justify investments.

Future Developments: Microsoft 365 Agents SDK

• The forthcoming Microsoft 365 Agents SDK will facilitate seamless integration of code elements between Co-Pilot Studio and other platforms, enhancing flexibility in agent development.

• Regardless of the chosen approach—whether extending Microsoft 365 co-pilots or creating custom agents—the decision should align with resource availability, expertise level, and desired control over the project.

Final Thoughts on Agent Development Choices