Foundry IQ: Camada de conhecimento para agentes de IA

Introduction Welcome to the Live Session

• Thank you for joining today's live session.

• Please read our Code of Conduct to promote a friendly and respectful environment.

• The session is being recorded and will be available on YouTube in 48 hours.

Purpose of the Session Understanding AI Agents

• Today's goal is to demonstrate how to build agents that understand environments and retrieve information securely.

• Technical terms are easy to identify, with perfect spelling aiding comprehension.

• Users often struggle with formulating precise questions for AI.

Real-Life Example Technician's Query

• A technician describes an issue with a component, mixing clues about a power supply problem.

• Multiple questions arise from the technician's interpretation of the situation.

• Equipment manuals and company policies are essential for addressing such queries.

Knowledge Integration Unifying Information Sources

• To answer effectively, we need to unify internal sources like manuals and web consultations while managing noise in queries.

• Foundry Q serves as a knowledge base for agents, eliminating the need for perfect user questions.

• Queries can be refined through organization based on provided information.

Foundry Interface Navigating Microsoft Foundry

• Overview of Microsoft Foundry's interface for developing generative AI applications.

• Emphasizes using diverse knowledge bases for efficient question answering through agents.

Contextual Retrieval Enhancing Question Efficiency

• Planning involves generating specific substitute questions to improve context retrieval efficiency.

• Distinguishing between equipment repair intentions and general knowledge enhances clarity in responses.

Final Response Elements

• Final answers include various technical elements identified during query processing.

• Identifies meanings behind indicators like warning lights on equipment.

Knowledge Sources Utilization

• Cites sources used by agents, including blogs detailing specifications relevant to user inquiries.

• Highlights how responses are generated without manual pipelines or multiple tools pointing at different knowledge sources.

Domain Knowledge Representation

• Knowledge bases represent complete domains such as sales, legal, technical support, etc.

• Different teams maintain fragmented information across various formats (PDF, images).

Challenges in Centralized Knowledge Systems

• Instead of centralized systems, reliance on individual expertise complicates access to diverse data structures.

• Each data function requires unique access methods depending on user queries.

Agent Responsibilities & Architecture Separation

• Logic must reside within agents rather than relying solely on external systems for information retrieval.

• Fondre Q contrasts traditional engineering by separating responsibilities within its architecture design.

Knowledge Base and Agent Functionality

• Agents focus on tasks, instructions, and tools while the knowledge base understands the domain.

• Knowledge sources encapsulate information needed for queries, detailing how to access data.

• Important change in FondreQ allows for non-indexed data sources.

Data Processing Strategies

• Non-indexing is useful when copying external content doesn't make sense, like web data.

• Results from both indexed and unindexed sources are merged to prioritize relevant outcomes.

Creating a Knowledge Base

• Demonstration of adding a knowledge source within an agent setup.

• Three storage containers created: repair policies, machine specifications, and diagrams.

Content Extraction Techniques

• Standard extraction method used for documents with Mociar.

• Content Understanding service available in GA extracts textual content from images.

Interacting with the Knowledge Base

• Create a knowledge base; process may take time but is essential for functionality.

• Instructions provided to assist technicians in the field with tool descriptions and behavior rules.

Querying the Knowledge Base

• Ability to ask questions based on the knowledge base after setup completion.

• Responses are classified for relevance before presenting them to users.

Independence of Knowledge Sources

• Knowledge bases can evolve independently; sources can be added or removed as needed.

Integration with Azure AI Search

• Fondre IQ built on Azure AI Search enhances document retrieval efficiency and scalability.

API Usage Options

• Clients can choose between REST API or SDK options (Python, Java).

Exploring Created Knowledge Bases

• Users can start with either Fondre Q or search APIs based on preference.

Accessing via VS Code

• Demonstration of accessing knowledge bases through VS Code using Python classes.

API Experience and Knowledge Base Integration

• Demonstrates the use of classes and search constructs while executing an API experience.

• Discusses extending knowledge sources using server smsp, returning to the phone interface for practical demonstration.

• Introduces new constructions for connecting with multiple knowledge bases, querying directly through WCA in Python.

Query Planning and Execution

• Observes activity tracking of query recovery, noting initial planning phases.

• Highlights that questions are sent to various knowledge sources followed by another planning phase to assess context sufficiency.

• Mentions explaining how to halt cycles based on previous iterations' contexts for final responses.

Repository and Incident Association

• References a Git repository containing various axes related to the software being discussed.

• Shows how responses link incidents of software associated with components, allowing detailed activity review from IPA's responses.

Understanding Agent-Based Search

• Explains agent-based search layers above knowledge sources that facilitate question generation and result reordering.

• Describes the first step as query planning where content is analyzed to determine optimal strategies.

• Details source selection for launching queries across configured sources.

Result Classification and Iteration Process

• Discusses classification applied upon receiving results from sources, ranking candidates for answering questions.

• Indicates further assessment occurs post-classification to determine if sufficient information is available for answers.

• If confidence is low, the process iterates again; otherwise, it concludes.

Evaluating Effort vs. Latency in Search

• Analyzes latency within the complete pipeline involving interactions assessing information sufficiency for user queries.

• Compares outcomes based on varying effort levels: minimum, medium, or maximum effort during execution of pipelines.

Microsoft Study on Agent Performance

• Introduces a Microsoft study evaluating whether agent-based searches outperform direct index searches in efficiency and quality.

• The study aims to answer if additional effort improves response relevance compared to brute force methods.

Results from Performance Evaluation

• Reports a 36% average increase in response relevance when using higher effort levels in agent-based retrieval across datasets.

• Highlights significant improvements observed in some datasets reaching up to 60% relevance enhancement.

Creation of Multiple Indexes

• Instead of a single index, nine indexes were created for better relevance, achieving 89% accuracy with minimal effort.

• The speaker invites questions and confirms time availability for discussion.

• Demonstration begins on how to access the knowledge base in Fondre.

Connecting to Storage

• The project is connected to storage where documents are located.

• Emphasis on choosing an effective extraction method that handles various document types like images and tables.

• Discusses creating an Analyze Source for MVP using Fondre IQ.

Configuring Search Parameters

• Configuration of search parameters is explained, focusing on determining question handling and iteration efforts.

• A summary generation process is described, favoring textual content without changes.

• Indexing process is ongoing; searching should trigger indexing.

Managing Indices

• Introduction of Machine Visual Index as a newly created index.

• Documents have been indexed in storage, enabling future searches within Fondre.

Agent Creation and Knowledge Base Update

• An agent is being created with demo.signature.re2 for generating responses.

• Knowledge base update allows querying through the agent's tools.

Final Remarks and Q&A Session

• Waiting for knowledge base updates before proceeding with queries from the audience.

• Summary of the session; waiting for knowledge base updates to be used directly from the index.

• Closing remarks express gratitude for participation and offer further resources.