AutoGen Tutorial 🚀 Create Custom AI Agents EASILY (Incredible)

Introduction to Autogen

In this section, the speaker introduces Autogen, a project by Microsoft that allows users to create autonomous agents and have them work together. Autogen is a flexible framework that enables the orchestration, optimization, and automation of large language model workflows.

Autogen Features and Benefits

• Autogen is a project by Microsoft that enables the creation of autonomous agents working together.

• It offers customizable and conversational agents that leverage advanced language models like GPT-4.

• Autogen can be dropped into any project and serves as a drop-in replacement for OpenAI API.

• Multiple AI agents working together tend to produce better outputs in various tasks such as coding or creative writing.

• Autogen simplifies the orchestration, optimization, and automation of large language model workflows.

How Autogen Works

This section explains how autogen works and its capabilities in building complex multi-agent conversation systems.

Defining Agents and Roles

• With autogen, building a complex multi-agent conversation system involves defining a set of agents with specialized capabilities and roles.

• For example, an engineering team could consist of agents playing roles such as engineer, project manager, quality assurance, etc.

Interaction Behavior between Agents

• In addition to defining agents and roles, autogen allows users to define how these agents interact with each other.

• Users can specify what an agent should reply when receiving a message from another agent.

Default Agents in Autogen

This section discusses the default agents provided by autogen for building enhanced chat systems.

User Proxy Agent

• The User Proxy Agent acts on behalf of the user in the conversation system.

• It can make decisions autonomously or ask the actual human user for input.

• It can also execute code on its own or seek approval from the user before running it.

Assistant Agent

• The Assistant Agent is a default agent in autogen and plays the role of an AI assistant.

• It can write code and perform tasks assigned to it within the conversation system.

Autogen's Interaction with Humans

This section explains how autogen seamlessly integrates with humans and allows for human feedback and intervention.

Human Feedback and Intervention

• Autogen automates chat between agents while allowing human feedback or intervention when necessary.

• The User Proxy Agent engages humans, simulates user behavior, and uses tools appropriately.

• Users can provide input at any point during the conversation process.

Examples of Using Autogen

This section mentions that there are examples available on GitHub for using autogen.

Automated Task Solving with Code Generation, Execution, and Debugging

• One example provided by Microsoft demonstrates how to use autogen to automate task solving with code generation, execution, and debugging.

• The example showcases how the Assistant Agent and User Proxy Agent work together to write and execute Python code.

For more detailed information on using autogen, refer to the examples provided on GitHub.

The Beauty of the Multi-Agent Framework User Proxy Agent

In this section, the speaker explains the concept of a user proxy agent in a multi-agent framework. The user proxy agent acts as a proxy for the human user to execute code written by the assistant agent or automatically execute code.

• The user proxy agent can run code on behalf of the human user.

• It can either ask for approval from the user before running the code or automatically execute it based on settings.

• The human input mode setting determines whether approval is required or not.

• The user proxy agent can solicit feedback from the human user or provide auto feedback based on code execution results.

• The assistant agent will debug the code and suggest new code if there are errors.

• Communication between agents continues until the task is completed.

Setting Human Input Mode and Max Consecutive Auto Reply

This section focuses on setting up parameters related to human input mode and maximum consecutive auto reply in the multi-agent framework.

• Human input mode options include "always" (asking for input at every step), "terminate" (asking for input only during termination), and "never" (no input required).

• Max consecutive auto reply determines how many back-and-forth interactions are allowed before terminating the task.

• These settings help control when and how much interaction is needed from the human user.

Installing Pi Autogen Library

Here, we learn about installing the Pi Autogen library, which is used in this example.

• To install Pi Autogen library, use pip: pip install pi-autogen.

• This library enables interaction with OpenAI's GPT models through an API endpoint.

Configuring API Endpoint and Importing Autogen Library

This section covers configuring the API endpoint and importing the Autogen library.

• The API endpoint can be customized to use different models.

• The autogen library is imported to interact with the GPT models.

• A configuration list is created from a JSON object, specifying the model(s) and API key(s) to be used.

Defining Configuration List for Autogen

Here, we define the configuration list for Autogen, which includes the model options and corresponding API keys.

• The configuration list is created using autogen.config_list_from_json() function.

• Multiple API endpoints can be specified in the config list, allowing the use of multiple models.

• Each model is associated with an API key obtained from OpenAI.

Setting Up OpenAI Account and Obtaining API Key

This section explains how to set up an OpenAI account and obtain an API key required for using Autogen.

• To create an OpenAI account, visit their website and sign up.

• In the account settings, navigate to the API key section.

• Create a new key with a name (e.g., "autogen").

• Copy the generated API key for later use.

Adding Config List JSON Object and Running Code

This section involves adding the config list JSON object into code and running it.

• The previously defined config list JSON object is added to the code block.

• This ensures that it can be read correctly by Autogen.

• After running this code, Autogen will import successfully with configured settings.

Example Task: Checking Stock Price Change

In this example task, we demonstrate how to check stock price changes using Autogen.

• The task involves writing code to fetch stock prices and present them to the user.

• Two agents are defined: the assistant agent and the user proxy agent.

• The assistant agent is configured with seed, config list, and temperature settings for generating responses.

• The user proxy agent represents the human user and has its own configuration.

• The human input mode is set to "never," meaning code execution occurs without asking for approval.

• The task is initiated by the user proxy agent with a specific message.

Configuring Assistant Agent and User Proxy Agent

This section focuses on configuring the assistant agent and user proxy agent for the example task.

• The assistant agent is configured using autogen.AssistantAgent class.

• Seed 42 is used for caching prompts to avoid repeated calls to GPT models.

• Config list, temperature, and other settings are passed to the assistant agent.

• The user proxy agent represents the human user and has its own configuration.

Initiating Chat with Assistant Agent

Here, we initiate a chat between the user proxy agent and the assistant agent.

• The dialogue starts with the user proxy initiating chat with an assistant using a specific message.

• Multiple agents can be defined to work together in a multi-agent framework.

Timestamps may not be available for all sections.

Getting the Current Date and Year-to-Date Gain for Stocks

The transcript discusses how to use Python to get the current date and calculate the year-to-date gain for stocks using the Y Finance library.

Getting the Current Date

• The assistant suggests using Python to get the current date.

• Python code is provided to retrieve the current date using date.today().

Calculating Year-to-Date Gain for Stocks

• The assistant recommends using the Y Finance library in Python to obtain stock prices.

• Users are instructed to install the library by running pip install yfinance in their terminal.

• Python code is provided to calculate the year-to-date gain for Meta and Tesla stocks.

• An error occurs during execution, which is passed back from user proxy to assistant.

• The assistant acknowledges the oversight and provides corrected code.

• The updated code successfully calculates and prints the year-to-date gain for Meta and Tesla stocks.

Displaying Stock Price Change Year-to-Date Chart

This section explains how to plot a chart of stock price changes year-to-date using matplotlib library in Python.

Plotting Stock Price Change Chart

• The user requests assistance in plotting a chart of stock price changes year-to-date and saving it as "stock_price_yt.png".

• The assistant suggests using matplotlib library in Python for plotting charts.

• Instructions are given on installing matplotlib by running pip install matplotlib.

• Code is provided for importing matplotlib, plotting data, and saving it as an image file.

• An error occurs due to undefined variables (metadata and Tesla data).

• The assistant acknowledges the oversight and provides corrected code.

• After successful execution, a generated file named "stock_price_yt.png" is displayed.

Teaching AI New Skills with Autogen

This section demonstrates how to use Pi Autogen to teach AI new skills through natural agent interactions without requiring knowledge of a programming language.

Setting up Autogen

• The user is instructed to install Pi Autogen by running pip install piautogen.

• API endpoint and autogen are imported, and the llm config is set up.

Example Task: Literature Survey

• An example task of conducting a literature survey is introduced.

• The goal is to find research papers on a specific topic, categorize application domains, and plot a bar chart of the number of papers in each domain.

• Agents are constructed for this task.

The transcript continues with further content, but it exceeds the recommended length for a single section. Therefore, it should be divided into additional sections for better organization and readability.

Finding ARXIV Papers on Trust Calibration in AI-based Systems

The assistant demonstrates how to find ARXIV papers related to trust calibration in AI-based systems using the ARXIV API. The process involves writing code to scrape the necessary information and fetching a list of papers with their titles, authors, summaries, and links.

Finding ARXIV Papers

• The assistant includes the code directly in the script instead of loading separate files.

• The user initiates the chat and the assistant starts working on finding ARXIV papers on trust calibration in AI-based systems.

Scraping Paper Information

• The assistant writes code to use the ARXIV API for fetching paper details.

• It successfully scrapes information about several papers related to trust calibration in AI-based systems.

• The python script fetches titles, authors, summaries, and links for these papers.

Analyzing Application Domains

• Task two involves analyzing the results obtained from the previous step.

• The assistant uses paper summaries to determine the application domains studied by these papers.

• It is unclear how exactly it analyzes the papers but it might rely on reading their summaries.

Generating a Bar Chart

• Task three requires generating a bar chart showing application domains and the number of papers in each domain.

• Matplotlib library is used along with Python for creating this chart.

• A script is provided by the assistant for generating the bar chart.

Creating a Reusable Recipe

After completing multiple tasks, including finding papers and generating a bar chart, there is an option to create a reusable recipe. This recipe contains all necessary steps with well-documented Python functions that can be used for similar tasks in the future.

Reflecting on Sequence

• Task four involves reflecting on all completed steps and creating a recipe for future use.

• The assistant suggests creating well-documented Python functions to perform similar tasks in the future.

• It emphasizes separating coding and non-coding steps into different functions.

Creating a Recipe

• The user requests to create a reusable recipe for the sequence of steps followed.

• The assistant lists out the sequence of steps and provides all the necessary code as part of the recipe.

• Python functions for fetching papers from ARXIV and generating a bar chart are defined successfully.

Conclusion

The assistant concludes by highlighting the incredible capabilities demonstrated in this video. It encourages viewers to share their use cases and requests for further exploration. Reusing recipes is mentioned as an option, but it is skipped in this case.