Copilot Studio 101

Copilot Studio 101: Getting Started

Introduction to Copilot Studio

• The speaker introduces the session as an entry-level course on Copilot Studio, promising a comprehensive overview for beginners.

• Emphasizes that this tutorial will provide detailed insights to help users start building experiences in Copilot Studio effectively.

Event Sponsorship

• Mentions sponsorship by the AI and Co-Pilot Summit, scheduled for March 17th-19th at Tory Pines Hilton, San Diego.

• Highlights participation from over 750 business and technology leaders, including notable speakers from Microsoft and the community.

Understanding Agents

Definition of Agents

• Defines agents as an evolution of business process automation and conversational bots, enhanced by generative AI technology.

Types of Agents

• Warns against starting with autonomous agents due to their complexity; suggests beginning with simpler types like retrieval agents.

Retrieval Agents

• Describes retrieval agents as similar to ChatGPT but grounded in specific datasets, allowing for data-driven conversations.

Task-Based Agents

• Explains task-based agents' capabilities to interact with back-end systems and APIs, enabling more complex functionalities within Copilot Studio.

Autonomous Agents vs. Business Process Automation

• Clarifies that autonomous agents differ from traditional business process automation; they can make independent decisions based on provided knowledge and tools.

Overview of Copilot Studio

What is Copilot Studio?

• Introduces Copilot Studio as Microsoft's tool for building standalone agents tailored for enterprise scenarios.

Versions of Copilot Studio

• Discusses two versions: full Copilot Studio (for enterprise development with policy controls) and Copilot Studio Light (for citizen developers).

Full vs. Light Version Features

• Full version allows extensive control over data usage and developer access while the light version focuses on creating declarative agents limited to accessible data within M365 environments.

Understanding Copilot Chat and Studio

Overview of Copilot Chat

• Copilot Chat Implementation: Designed for organizational use, providing a secure environment for inquiries about source code without exposing sensitive information to the internet.

• Safety Features: Unlike general web-based chat solutions, Copilot Chat ensures that user data is not used for model training, maintaining confidentiality within enterprise settings.

M365 Copilot Capabilities

• Enhanced Functionality: M365 Copilot integrates with Office Graph to facilitate personal productivity and access corporate data, allowing responses based on internal resources rather than just web content.

• Extensibility with Microsoft Agents: Users can enhance their experience by creating custom agents using Microsoft tools, adding tailored functionalities beyond default capabilities.

Building Agents with Copilot Studio

• Agent Development Process: Agents can be built using either full or light versions of Copilot Studio, enabling both comprehensive and simplified experiences.

• Integration Flexibility: Full Copilot Studio allows users to create standalone agents or extend existing first-party products without limitations.

Orchestration in Copilot Studio

• Conversational Orchestrator Role: Functions like a router in IP networks, facilitating communication between various components and enabling robust agent development across different domains (e.g., IT, HR).

• Cross-Domain Applications: The ability to build versatile agents such as employee assistants showcases the potential for broad applications beyond single-function roles.

Utilizing Azure AI Foundry

• Model Fine-Tuning and Integration: Azure AI Foundry complements Copilot Studio by allowing users to fine-tune models and access industry-specific resources.

• Combined Tool Usage Recommendation: Leveraging both platforms is encouraged for optimal agent development; each serves unique purposes that enhance overall functionality.

Third Party Integrations

• External System Compatibility: The orchestration capabilities of Copilot Studio enable integration with third-party systems through protocols like Ato (agent-to-agent protocol).

• Trust-Based Integration Approach: Users can incorporate external technologies into their workflows as long as they trust the sources involved in the orchestration process.

Core Components of Copilot Studio

• Topics as Advanced Control Mechanism: Topics allow creators to dictate specific responses and prioritize internal knowledge bases over external ones when necessary.

Understanding Copilot Studio Features

Introduction to Topics in Copilot Studio

• The ability to integrate various functionalities within Copilot Studio allows for the creation of tailored experiences, giving users control over generative AI deployment.

• Topics are highlighted as a powerful feature in Copilot Studio, essential for achieving specific objectives during development.

Use Cases for Scripted Responses

• An example is provided where a major soda manufacturer requires scripted responses to ensure marketing messages are consistent and legally compliant.

• The emphasis on topics as a "superpower" within Copilot Studio illustrates their importance in maintaining control over generated content.

Knowledge Management with RAG Patterns

• Knowledge management utilizes a Software as a Service (SaaS)-based retrieval augmented generation (RAG) pattern, which enhances data interaction capabilities.

• Users can leverage existing data sources like SharePoint or uploaded files without needing extensive custom builds, streamlining the process of answering queries.

Tools and Their Capabilities

• Tools enable the construction of conversational interfaces over APIs and facilitate business process automation, enhancing functionality within applications.

• Custom prompts and integration with external components (e.g., MCP servers) are key features that empower developers to create dynamic interactions.

Analytics Integration

• Significant investments have been made in analytics capabilities within Copilot Studio, providing insights necessary for managing deployed agents effectively.

Communication Channels Overview

• Multiple communication channels (phone, web chat, Teams, etc.) can be integrated into Copilot Studio to enhance user engagement across platforms.

• Each channel has unique requirements; understanding these is crucial for effective implementation and maximizing functionality.

Understanding Variables vs. Memory

• Variables manage conversation state by storing relevant data during interactions; this differs from memory used in generative AI models.

• Examples illustrate how variables can store user information to tailor conversations based on context (e.g., employee status).

Building Agents in Copilot Studio

• Once all components are understood, users can initiate agent creation through simple commands that provision necessary resources automatically.

Understanding Copilot Studio's Functionality

Overview of Knowledge and Experience

• The process begins with acquiring knowledge to create desired experiences, focusing on specific topics for user control.

• Actions, previously known as tools, are integrated with back-end systems and APIs to facilitate task-based scenarios.

Multi-Channel Publishing and User Interaction

• Content is published across multiple channels to effectively engage users where they are located.

• Continuous monitoring and improvement through analytics are essential for optimizing user interactions.

Orchestrator Role in Copilot Studio

• The orchestrator determines the appropriate use of topics, knowledge, and tools based on user needs.

• Agents within Copilot Studio can be tailored for specific scenarios (e.g., HR or IT), enhancing functionality.

Integration with External Systems

• The rebranded M365 agents SDK allows building external entities that interact with Copilot Studio.

• Azure Foundry components enable the creation of full agents that can be called upon as needed.

Generative Orchestration Layer Explained

Power of Generative Orchestration

• The generative orchestrator analyzes all available resources when a user query is made to generate an actionable plan.

• This plan may involve various combinations of topics, actions (tools), and knowledge to provide comprehensive answers.

Unified Response Mechanism

• Responses from different tools are consolidated into a single unified response layer for clarity and coherence.

Fallback System Implementation

• If the planner cannot resolve a query, it triggers a fallback system that may escalate issues to live agents if necessary.

Deep Dive into Knowledge Management

Knowledge Framework in Copilot Studio

• Knowledge operates on a SaaS-based RAG pattern allowing questions over accessible data sources like SharePoint or Dynamics 365.

Moderation Before Query Processing

• A moderation step ensures inappropriate queries (e.g., harmful requests) do not reach AI models; this is configurable within Copilot Studio.

Query Optimization and Information Retrieval Process

Query Optimization Stage

• The second stage involves crafting a query that effectively communicates the user's intent. For example, asking "How long is it?" about a rake needs to be reformulated to specify which rake is being referred to.

• This stage emphasizes understanding conversational context to build an optimal query for backend systems, ensuring accurate information retrieval.

Information Retrieval Stage

• The third stage focuses on how data is retrieved from various sources, clarifying that Copilot Studio does not automatically ingest data but relies on existing search services.

• The quality of information retrieval depends significantly on how data is indexed and where it resides (e.g., Bing for web searches or SharePoint for internal queries).

Summarization Process

• In the fourth stage, retrieved search results are passed into a summarization process where large language models synthesize the information based on user queries and conversation context.

• Microsoft implements responsible AI guardrails during this process to ensure that summaries provided do not include inappropriate content.

Providence Validation

• Stage five involves validating that the responses generated are grounded in the original data requested by users, creating citations as necessary.

Summary Moderation

• In stage six, responses from the model undergo moderation checks to confirm appropriateness and accuracy before being presented back to users.

Tools and Capabilities in Copilot Studio

Overview of Tools

• Tools enable integration with APIs and advanced functionalities beyond basic patterns. They include custom prompts and connections to various servers.

Types of Tools Available

• Prompts: These allow users to provide instructions for different models (e.g., Llama or GPT variants), facilitating diverse applications within Copilot Studio.

• Flows: Business process automations created using Power Automate help manage inputs and outputs efficiently within workflows.

Computer Use Concept

• Computer Use: This concept allows interaction with applications lacking APIs through virtual machines that automate UI tasks; currently in preview status.

Custom Connectors & REST APIs

• Custom Connectors: Over 1,500 pre-built connectors are available for immediate use alongside options for building custom ones tailored to specific needs.

• REST APIs: Users can integrate their existing RESTful APIs into Copilot Studio seamlessly.

Model Context Protocol (MCP)

• MCP enables integration with model context protocol servers, enhancing functionality within Copilot Studio's ecosystem.

Understanding Topics and Triggers

Engagement Triggers in Topics

• A trigger initiates engagement with a topic; various types exist beyond standard triggers, allowing flexibility in how topics respond based on user interactions.

Advanced Features of Copilot Studio

Overview of Advanced Mode

• The advanced mode allows users to define when an agent should be activated, providing a description for its usage.

• Users can create reusable topics that will only activate when called by another topic, enhancing modularity in design.

Middleware Functionality

• There is potential to intercept responses generated by the system, allowing for processes like checking for personally identifiable information before sending responses.

• This feature emphasizes the power of triggers within the system, enabling customized actions based on specific events.

Customization and Control

• Users have complete control over customization within Copilot Studio, ensuring predictable behavior when engaging with specific topics.

• The session transitions into a practical demonstration of creating a new agent within Copilot Studio.

Creating Agents in Copilot Studio

Agent Creation Process

• The creation process is straightforward; users can enter necessary information directly or utilize advanced settings for language and schema adjustments.

• A blank agent can be created through an advanced option, but users are encouraged to build agents tailored to their needs.

Provisioning and Setup

• After initiating the setup, there may be a delay as the platform provisions various components required for the agent's functionality.

• Once provisioning is complete, users receive confirmation that their agent has been set up successfully.

Initial Configuration Options

• Users can edit descriptions and instructions provided with their newly created agents to better align them with intended functionalities.

• Different models are available for selection during setup, allowing further customization based on user requirements.

Integrating Knowledge into Agents

Adding Knowledge Sources

• Users can ground their agents' knowledge by integrating external sources such as public websites (e.g., learn.microsoft.com).

Querying Capabilities

• With integrated knowledge sources, users can ask questions about using Copilot Studio and receive accurate answers supported by citations from those sources.

Reducing Hallucinations in Responses

• To minimize irrelevant outputs ("hallucinations"), users can adjust settings to restrict responses solely to provided data rather than general web knowledge.

Ensuring Data Relevance

• By disabling general knowledge options in settings, agents will focus exclusively on user-defined content while answering queries.

Licensing Guide and Tool Creation in Copilot Studio

Uploading and Indexing Files

• The speaker discusses uploading a licensing guide for Copilot Studio to Data Verse, which will index the file for searchability.

• Once uploaded, the file enters an "in progress" state until indexing is complete, after which it changes to "ready."

• With the file indexed, users can ask questions about the licensing model of Copilot Studio, with citations provided from the document.

Utilizing Retrieval Agent Patterns

• The process described is part of what is known as retrieval agent (RAG) patterns, allowing users to query information effectively.

• The speaker expresses interest in exploring more complex functionalities beyond simple queries.

Exploring Tools within Copilot Studio

• Users can view available tools and navigate to a dedicated tools page; there’s an option to add new tools.

• A demonstration begins on creating a simple tool using a weather API connector that does not require user authentication.

Configuring API Interactions

• The speaker configures how to interact with the weather API by modifying input parameters such as location and units.

• Descriptions help define when this tool should be used; additional details allow switching between end-user credentials and maker-provided credentials.

Dynamic Input Handling

• Inputs like location are dynamically filled by AI; users can customize these inputs based on their needs.

• Units can be set (e.g., Imperial or Metric); in this case, Imperial is chosen for consistency in responses from the API.

Testing the Weather Tool

• After saving configurations, users can test the tool by asking about current weather without providing a location initially.

• The system prompts for missing information (location), demonstrating its interactive capabilities. Upon providing "Orlando," it retrieves relevant weather data successfully.

What is Atmospheric Pressure?

Understanding API Interactions

• The discussion begins with a question about atmospheric pressure, specifically in Orlando, highlighting the capability of an API to provide contextual answers.

• Emphasizes the value of having an interactive API that can respond to user queries in a conversational manner.

Tool Development Using Prompts

• Transitioning from connectors to tools built on prompts, indicating a shift in focus towards more complex functionalities.

• Introduction of a prompt builder where users can create tools designed for specific tasks, such as analyzing prompts.

Creating and Testing Tools

• A prompt named "prompt analyzer" is created to evaluate the quality of user-provided prompts.

• Users are encouraged to input sample data for testing their prompts before finalizing them.

Model Selection and Validation

• Users have access to various models, including custom ones from Azure AI Foundry, allowing flexibility in tool functionality.

• The importance of validating responses through testing different models is highlighted before saving the tool configuration.

Finalizing Tool Configuration

• After saving the tool, it becomes available for use; additional configurations are necessary for effective operation within Copilot Studio.

• The need for clear instructions on when and how to call the tool is emphasized, ensuring proper usage without requiring user authentication.

Enhancing Agent Instructions

• Modifications are made to agent instructions to clarify its functions regarding prompt analysis versus other capabilities.

• A method is introduced for disambiguating between different functionalities within Copilot Studio by editing agent instructions directly.

Testing Functionality

• The session concludes with a demonstration of testing the newly created prompt analyzer tool by asking it to summarize text.

Prompt Analysis and Custom Tool Creation

Analyzing Prompts for Improvement

• The speaker demonstrates how to analyze a prompt by requesting suggestions for improvement, showcasing the capabilities of the prompt analyzer.

• After running the analysis, the tool provides actionable feedback on enhancing the initial prompt based on user input.

Building Custom Tools with Copilot Studio

• The discussion shifts towards creating custom tools using prompts, emphasizing their power in tailoring functionalities.

• The speaker introduces an advanced mode for building specific actions within Copilot Studio, allowing users to have greater control over their implementations.

Creating Topics and Collecting User Information

• A new topic is created to collect user information for joining the Copilot Studio mailing list, highlighting the ease of use within Copilot Studio.

• Upon creation, a description is provided that outlines when to utilize this tool effectively, along with a summary of its functionalities.

Enhancing User Interaction

• Users can modify questions within topics; for example, making prompts more professional through simple commands.

• The ability to undo changes allows users flexibility in refining their interactions and improving overall implementation quality.

Testing Implementations and Advanced Features

• A test run is conducted where users are prompted to join the mailing list; it successfully collects necessary information from them.

• While 90% of the process is automated, additional steps like integrating Power Automate flows are suggested for full functionality.

Custom Topic Creation and Triggers

• Users can create topics from scratch with various triggers available beyond simple user inputs; these include event-based triggers and inactivity detection.

• The interface allows exploration of different triggers that enhance interaction capabilities significantly.

Adding Nodes and Advanced Functionalities

• Various nodes can be added such as sending messages or asking questions visually through adaptive cards.

• Conditions (if/then statements), variable management, and intent recognition are discussed as essential features that expand customization options in Copilot Studio.

Copilot Studio: Advanced Features and Variables

Overview of Copilot Studio Capabilities

• Copilot Studio allows users to select from existing connectors or tools, as well as create custom tools and agents for enhanced functionality.

• The advanced authoring mode in topics provides control over various capabilities, including sending HTTP requests and logging custom telemetry events.

Understanding Variables in Copilot Studio

• Variables enable the storage of information temporarily for later reference, differing from conversational memory which pertains to ongoing dialogue context.

• Types of variables include:

• Custom variables created by users.

• System variables that provide contextual data (e.g., last user input).

• Environment variables that change based on the operational environment.

Variable Scope and Persistence

• Variable scope determines how long a variable exists; common types include:

• Topic-level variables retain information only while on a specific topic.

• Global variables persist throughout the entire conversation.

• Cross-session variables allow data retention even after session timeouts, crucial for prolonged interactions.

Information State Management

• All variable data is stored within the conversation state, essential for agent-to-agent handoffs where state transfer may be necessary.

Agents in Copilot Studio

• Two types of agents can communicate within Copilot Studio:

• Child agents are embedded within parent agents, containing their own tools and knowledge while being part of a larger solution.

• This feature enhances modularity by allowing groups of knowledge to be reused across different agents.

Understanding Agents and Channels in K Copilot Studio

Child Agents and Connected Agents

• The speaker emphasizes the importance of child agents, describing them as an underrated feature within K Copilot Studio that allows for effective orchestration and packaging.

• Connected agents are introduced as independent entities that can be updated without manual intervention, which may have both positive and negative implications depending on user needs.

• Building connected agents can involve utilizing other copilot studio agents, fabric data agents, Azure Foundry, or the M365 copilot agent SDK.

• It is clarified that connected agents should not be viewed as modular components but rather as standalone applications requiring integration with careful consideration of conversation state transfer.

• Each connected agent will have its own authentication settings; thus, security measures must be established during setup to ensure proper interconnectivity.

Use Cases for Connected Agents

• An example is provided where an employee assistant integrates HR and IT capabilities while restricting access based on user roles (e.g., contractors needing IT support).

• The complexity of implementing connected agents versus their potential return on investment is discussed, highlighting the need for careful evaluation.

User Interaction Channels

• The concept of channels is introduced as a means for users to interact with their agents through various platforms like web chat, Teams, Facebook Messenger, etc.

• Copilot Studio supports APIs such as DirectLine for interaction; this API is well-documented and widely used in building agent interfaces.

Security Considerations Across Channels

• Different channels come with unique security requirements; for instance, using Entra ID authentication when connecting through Teams or M365 Copilot.

• Custom authentication may be necessary for specific channels due to varying capabilities across platforms.

Limitations of Channel Features

• Different channels support different features; voice channels may lack visual elements like adaptive cards.

• Developers must consider these limitations when designing interactions to ensure compatibility with the intended channel's capabilities.

Understanding Variables in Copilot Studio

Overview of Variable Management

• Discussion on the limitations of UDP regarding message order, particularly in multi-bubble scenarios, and its implications for text handling.

• Introduction to variables within Copilot Studio, focusing on collecting user responses such as email addresses.

• Explanation of topic-level variables and their settings, including options to receive or return values from other topics.

Global vs. Topic-Level Variables

• Description of global variables that persist across sessions and how they can be configured with descriptions for better context understanding.

• Mention of sensitive data management capabilities within variable settings, allowing redaction in logs for privacy.

Setting and Managing Variables

• Steps to create a node for variable management, specifically setting a variable value.

• Options available when setting a variable's value, including creating new variables or selecting existing ones from the topic or system level.

Utilizing Agents in Copilot Studio

Creating Child Agents

• Overview of agent creation options within Copilot Studio, including adding new child agents or connecting to external agents.

• Introduction to the care framework used for building prompts and its application in creating an agent focused on prompt guidance.

Implementation Details

• Process of naming and describing the newly created child agent aimed at enhancing prompt guidance using the care framework.

How to Engage and Configure an Agent

Configuring Agent Behavior

• The agent can be engaged with specific instructions, allowing customization of its behavior based on the provided information.

• Users can enhance the agent's knowledge by adding external resources such as websites or uploading PDFs for better context and understanding.

Adding Knowledge to the Agent

• Uploading files like PDFs provides additional knowledge that the agent can utilize, enhancing its ability to respond accurately.

• Prompt guidance is available to help users analyze their prompts effectively, ensuring that agents understand when and how to act.

Nudge Functionality for Clarity

• Users can edit instructions within the overview section to guide the agent’s actions more precisely, using a nudge feature for disambiguation.

• This nudging capability is optional but useful for clarifying when an agent should take specific actions.

Utilizing Sub Agents

• When asking questions about prompt guidance, the main agent may refer to a sub-agent specifically designed for that purpose, utilizing its specialized knowledge.

• This approach allows users to build a contextual framework of tools and knowledge tailored for various implementations.

Connecting External Agents

• Users have options to connect with other agents or external systems (e.g., Microsoft Foundry), which are currently in preview mode.

• Built-in agents within Copilot Studio provide limited control compared to external ones; however, they allow users to ask specific questions related to their functionalities.

Managing Channels and Authentication Settings

• Channels are accessible from an upper rail menu rather than being part of the building process; various channels are available for integration.

• Authentication settings can be customized according to user needs, including options like no authentication or Microsoft Entra ID for single sign-on capabilities.

M365 Copilot Channel Management

Managing Channels and Publishing Agents

• The speaker discusses the process of managing different channels within M365 Copilot, emphasizing the need to publish changes for agents.

• It is highlighted that forcing a manifest update may be necessary when updating an agent in Teams or M365 Copilot.

• The demonstration transitions to viewing the published agent within Teams using the web app interface.

Adding and Interacting with Agents

• The speaker successfully adds the created agent into Teams, making it available for interaction.

• After adding, users can initiate conversations with the agent; however, initialization may take some time before responses are generated.

• Users should wait for the conversation ID to be created before asking questions to ensure proper functionality.

Security Settings and Authentication

Connecting External Channels

• To connect external channels like Facebook Messenger, security settings must be adjusted to accommodate authentication providers not using Entra ID.

• Each channel has unique requirements that must be met for successful integration.

Analytics Overview in Copilot Studio

Understanding Agent Performance

• Analytics capabilities allow users to assess agent effectiveness, user satisfaction, and activity levels through various metrics.

• Admin analytics provide insights on usage patterns, license allocation, suggested actions, security reviews, and auditing capabilities via Purview.

Telemetry and Data Insights

• Full telemetry data is collected and stored in a telemetry store; upcoming demonstrations will showcase these analytics features.

• Published agents will begin populating analytics data once they have sufficient activity against them.

Evaluating Agent Effectiveness

Metrics for Success

• Users can calculate potential savings from their agents by analyzing performance metrics related to issue resolution rates (resolved vs. escalated vs. abandoned).

• Effectiveness assessments focus primarily on classic mode implementations; caution is advised when interpreting this data as it may not reflect all scenarios accurately.

Understanding Tool Capabilities in Copilot Studio

Overview of Tool Functionality

• The discussion begins with an exploration of various tools available for analyzing answers and knowledge generation, emphasizing the importance of understanding how these tools operate.

• Users can access detailed metrics on user reactions and satisfaction scores by session, providing insights into how well agents are performing based on user feedback.

User Feedback Mechanisms

• The significance of thumbs up and thumbs down ratings is highlighted as a method to gauge user satisfaction with deployed agents, contingent upon channel support.

• An introduction to evaluations is presented, which allows users to create test sets for assessing agent performance against predefined criteria.

Navigating the Power Platform Admin Center

• Upon launching Copilot Studio, users will encounter information regarding build messages and agent performance metrics, including security reviews and health monitoring.

• A comprehensive list of all agents within the user's environment can be accessed through the management section in Copilot Studio, offering a centralized view across multiple environments.

Utilizing Purview for Audit Management

• Instructions are provided on accessing Purview via purview.microsoft.com to manage audits effectively within the Power Platform.

• Users can create specific searches against audit logs by setting date ranges and selecting activities related to copilot operations.

Conducting Audits and Monitoring Performance

• The process for auditing copilot actions is explained, including options for filtering results based on specific activities like creation or deletion events.

• A demonstration shows how to initiate an audit search that tracks published items over time; however, it notes that processing may take considerable time (upwards of three hours).

Conclusion and Future Content

• The video wraps up with a summary of capabilities discussed regarding full audit logging in Copilot Studio. Viewers are encouraged to engage further by liking and subscribing for more content.