Why SharePoint Knowledge in Copilot Studio Isn’t Working (and How to Fix It)

Understanding SharePoint Integration with Co-Pilot Studio

Introduction to the Video

• The speaker addresses common issues users face when integrating SharePoint as a knowledge source in Co-Pilot Studio, emphasizing that many experience similar challenges.

• The video aims to explain how the integration works, its strengths and limitations, and provide solutions for better results.

Setting Up SharePoint Knowledge Source

• A typical scenario involves connecting to an internal SharePoint site containing documents like policies and information for staff access.

• The example used is a basic Human Resources site with standard policy documents on leave policies, flexible work, grievance procedures, and performance management.

• Additional content includes resumes from job applicants and an Excel spreadsheet listing candidates.

Creating an Agent in Co-Pilot Studio

• The process begins by creating an agent named "HR helper" within Co-Pilot Studio; the speaker assumes viewers are familiar with initial setup steps.

• Users should input the URL of their SharePoint site without "https" for proper configuration of the agent's knowledge base.

Adding Knowledge Sources

• Users can browse files directly or connect to document libraries; however, it's important to include other relevant information from the site.

• Descriptions added during this step help AI understand which sources it should reference later on.

Testing Agent Responses

• After setting up the HR agent pointing to the SharePoint site, users can test it by asking questions related to HR policies.

• If successful, the agent will respond accurately; otherwise, it may indicate uncertainty if questions aren't clearly found in documents or if content indexing is still processing.

Understanding AI Limitations

• New content may require time to index properly; patience is advised before expecting accurate responses from newly created agents.

• It's crucial to recognize that AI may generate answers based on general knowledge rather than specific organizational data when queries fall outside documented topics.

• This can lead to irrelevant or incorrect answers being provided instead of citations from internal documents.

Conclusion on Effective Use of Co-Pilot Studio with SharePoint

Understanding SharePoint Knowledge Management

Disabling General Knowledge for Effective Queries

• The speaker emphasizes the importance of disabling general knowledge in SharePoint use cases to improve query accuracy.

• After disabling general knowledge, the system is forced to acknowledge when it cannot answer a question, enhancing user experience.

Successful Query Examples

• Demonstrates effective queries such as "How do I create an about me post?" which retrieves information from the SharePoint site.

• Queries about employee leave policies successfully pull data from policy documents, showcasing the system's ability to access multiple sources for accurate answers.

Limitations and Challenges

• While simple questions yield good results, more complex or rephrased queries often fail due to mismatched keywords.

• The speaker illustrates this with examples where slight changes in wording lead to no results, indicating a need for precise language matching.

Understanding Knowledge Sources

• The speaker discusses three critical aspects that affect query success: understanding knowledge sources, managing expectations, and utilizing technology effectively.

• Suggestion to obtain at least one Microsoft 365 co-pilot license for improved indexing and better search results within SharePoint.

Retrieval Augmented Generation Explained

• Introduces the concept of retrieval augmented generation as a method where the system acts like an efficient librarian retrieving relevant information based on user intent.

• Clarifies that the system generates answers in its own words rather than copying text verbatim from documents.

Common Misconceptions

Understanding Microsoft 365 Co-Pilot and Its Limitations

Knowledge Retrieval vs. Learning

• The tool retrieves information based on provided knowledge but does not learn or mimic organizational styles, such as writing in a specific style guide.

• It can answer factual questions, like policies on the Oxford comma, by retrieving relevant information rather than generating content in a specified style.

Document Compatibility

• The tool works effectively with unstructured text-based documents like Word, PowerPoint, and PDFs but struggles with Excel data retrieval.

• Users may encounter issues when trying to extract lists or arrays from Excel; it is recommended to use structured databases for better results.

Licensing and File Size Limitations

• Microsoft 365 Co-Pilot has separate licensing from Co-Pilot Studio; users need to be aware of these distinctions.

• Without the appropriate license, users face limitations such as file size restrictions (up to 7 MB), which may hinder access to larger organizational knowledge bases.

Indexing Methods: Keyword vs. Vector

• Basic keyword indexing limits search capabilities; effective results depend on carefully constructed queries that match document language.

• With a Microsoft 365 Co-Pilot license, organizations benefit from enhanced file size limits (up to 512 MB for certain document types).

Semantic Understanding through Vector Indexing

• Vector indexing allows for semantic understanding by assigning numerical values to words based on meaning rather than just matching keywords.

• This method improves reasoning capabilities by grouping related terms together, enhancing the relevance of search results.

Practical Implications of Semantic Indexing

• Semantic understanding enables more nuanced responses; for example, searching "Apple" can yield contextually relevant results beyond just the word itself.

Enhancing AI Responses with Generative Orchestration and Vector Indexing

Introduction to Advanced Features

• The integration of vector indexing and semantic understanding allows for more nuanced questions, enabling sophisticated reasoning beyond literal keywords.

Generative Orchestration Feature

• Enabling the generative orchestration feature allows the language model to comprehend the intent behind questions, improving decision-making on which knowledge source to utilize.

• This feature enhances the agent's ability to navigate documents effectively, providing a toggle switch for activation. Despite being in preview mode, it offers significant improvements in response quality.

Improved Response Capabilities

• With generative orchestration enabled, the agent can now handle complex queries that were previously unmanageable, such as inquiries about parental leave.

• The system tracks sources of information better and provides relevant answers even when direct references are not found in initial documents.

Limitations with Excel Queries

• Despite advancements, querying Excel remains a challenge; users may still encounter limitations in obtaining satisfactory results from Excel-related questions.

Comparison of Answer Quality

• A comparison between responses generated with and without Microsoft 365 co-pilot reveals significant differences in understanding and sophistication when addressing complex employee role implications post-injury.

• The combination of generative orchestration and vector indexing leads to enhanced answer quality across similar queries.

Practical Examples of Enhanced Functionality

• When asking about family emergency leave options, the system accurately identifies various types of leave available due to improved semantic understanding.

• In scenarios involving multiple document versions (e.g., travel policies), confusion can arise; however, vector indexing helps pinpoint correct information from updated documents.

Conclusion on Document Management Strategies

How to Improve Your SharePoint Performance

Key Strategies for Enhancing SharePoint as a Knowledge Source

• Obtain Microsoft 365 Co-Pilot License: If possible, acquire at least one license for Microsoft 365 Co-Pilot in your tenant. This can significantly enhance performance, even if only one user has access.

• Enable Generative AI Orchestration: Activate the generative AI orchestration feature with or without the Microsoft 365 Co-Pilot. This feature is still in preview but is recommended for better responses.

• Understand Document Types and Sizes: Be aware of the types and sizes of documents you are querying. Large documents may only allow reading of the first part, which could limit effectiveness.

• Differentiate Between Data Structures: Recognize the difference between structured and unstructured data. Excel behaves like structured data but is not a database; understanding this distinction will improve how knowledge sources interact with your documents.