Introduction

In this section, Kevin Scott introduces himself and talks about the impact of AI on the world.

Kevin's Introduction

• Kevin Scott is introduced as the Chief Technology Officer and Executive Vice President of AI.

• Kevin thanks Satya for sharing an inspiring video that shows how technology is positively impacting the world.

• Kevin expresses his excitement to be at Build after a four-year hiatus and discusses how much has changed in the world of technology over these past four years.

• Kevin talks about his experience as a developer and how he has been chasing moments where something impossible became possible throughout his career.

• Kevin discusses how AI is helping people have those moments where they can take something in their hands, look at what was possible, what was impossible, and become inspired to do something great with it.

The Power of AI

In this section, Kevin talks about technological themes driving progress in AI.

Foundation Models

• There is an incredible amount of attention being paid to what's happening with the rapid progress with foundation models in particular.

• OpenAI's partnership with Microsoft is setting the pace of innovation in the field of AI right now.

End-to-end Platform for Building AI Applications

• Microsoft has an end-to-end platform for building AI applications that starts with Azure.

• The platform includes super powerful computers to train models from scratch and build applications on top of that infrastructure.

Copilot

• Kevin discusses the idea of a Copilot, which is an application that uses modern AI with a conversational interface to assist with cognitive tasks.

• Microsoft has built a platform for building copilots and is sharing patterns that have helped them build copilots.

Greg Brockman on Building GPT4 and ChatGPT

In this section, Kevin Scott interviews Greg Brockman, the President and Co-founder of OpenAI, about his experiences building GPT4 and ChatGPT.

Building ChatGPT

• ChatGPT was a challenging engineering project that required attention to detail from both an infrastructure and machine learning perspective.

• The team had been working on the idea of having a chat system for years before developing ChatGPT.

• The moment that really clicked for Brockman was when they trained GPT4 to follow instructions. This showed that the model was capable enough to have conversations.

• Once they realized that their infrastructure could support earlier models like GPT3 and newer models like GPT4, they knew they had to get ChatGPT out.

Developing GPT4

• Developing GPT4 was a labor of love for OpenAI after multiple failed attempts to surpass the performance of GPT3.

• To build GTP4, OpenAI went back to the drawing board and rebuilt their entire infrastructure with attention to every detail.

• Paying attention to every single detail is what led to success in building GTP4.

Empowering Developers with Plugins

• Satya Nadella talked about empowering developers with plugins during his talk at Build 2021.

• This shared approach will allow developers in the room to write software that can extend the capability of things like ChatGPT and other copilots being built by OpenAI.

OpenAI and the Future of AI

In this section, Greg Brockman and Kevin Scott discuss the potential of OpenAI's technology to improve systems for everyone. They also talk about how developers can leverage this technology through open standards.

The Power of Open Standards

• OpenAI's technology is an amazing opportunity for every developer to leverage in a way that makes systems better for everyone.

• By designing it as an open standard, any AI can use it, allowing developers to build something once and have anyone leverage it.

• This core design principle allows any developer who wants to plug in and get the power of the system to bring all of the power of any domain into ChatGPT.

Pushing the Limits of Technology

• Greg Brockman talks about how he constantly thinks about pushing the limits of technology at OpenAI.

• With GPT4, they are in an early stage of really pushing it, with vision capabilities that have been announced but still being productionized.

• Over the past couple of years, there has been a 70% price reduction two years ago and a 90% cost reduction this past year. They hope to do the same thing repeatedly with new models.

Making AI Work in Specific Domains

• While technology is getting better and better, every developer can add value by going into specific domains and figuring out how to make this technology work there.

• Companies that are in legal domain can benefit from getting expertise and talking to lots of lawyers to understand their pain points with this technology.

Copilots and the Future of AI

In this section, Kevin Scott talks about how ChatGPT fits into the Copilot pattern and how the idea of a copilot is actually pretty general.

The General Idea of a Copilot

• The idea of a copilot is actually pretty general, where you have a multi-turn conversational agent-like interface on your software that helps you do cognitively complex things.

• This applies to more than just helping someone do software development. There are search copilots, security copilots, productivity copilots, and many more.

ChatGPT as a Copilot

• ChatGPT fits into the Copilot pattern along with Bing chat, GitHub Copilot, Microsoft Security Copilot, Microsoft 365 Copilot, and Designer.

• ChatGPT can help developers do cognitively complex things by providing suggestions for code completion or generating entire blocks of code based on natural language input.

Copilot Technology Stack

In this section, the speaker talks about the importance of platforms and how they allow developers to build more ambitious things. He also introduces the Copilot technology stack and explains how it enables them to move quickly with safety.

The Importance of Platforms

• Platforms are important because they allow developers to build more ambitious things than they otherwise would be able to.

• Building everything from scratch is economically infeasible, so platforms like Copilot are necessary.

• The fact that foundation models and the entire platform are reusable and generalizable is a fantastic thing.

Plugins

• Plugins are powerful mechanisms that can be used to augment a copilot or an AI application so that it can do more than what the base platform allows you to do.

• Plugins help augment AI systems so that they can access APIs and perform arbitrary computations safely on behalf of users.

• Plugins act as actuators of the digital world, allowing copilots to connect with anything that can be done digitally.

Anatomy of a Copilot

In this section, the speaker discusses what a copilot looks like, what is shared among all built copilots, and what platform components are being built for building copilots.

User Experience

• Building Copilot user experiences requires understanding unmet user needs and applying technology appropriately.

• Safety and security must be considered from the very first steps of building copilot applications.

Building a Great Product

• Building a great product is essential and requires understanding unmet user needs and applying technology appropriately.

• The model is not the product, unless you are an infrastructure company.

Using Infrastructure to Solve Problems

In this section, the speaker emphasizes the importance of using existing infrastructure to solve problems instead of building new infrastructure. They also encourage building great experiences that delight users.

Key Points

• Use existing infrastructure to solve problems.

• Don't build unnecessary infrastructure.

• Build great experiences that delight users.

Getting Feedback and Iterating Quickly

The speaker emphasizes the importance of getting feedback from users as quickly as possible and iterating on it to make improvements.

Key Points

• Get things out to users quickly.

• See what works and what doesn't work.

• Iterate and make improvements.

Copilot Stack Structure

The speaker introduces the Copilot stack structure, which consists of three boxes corresponding to the front end, mid-tier, and back end of a normal application. They mention that subsequent talks will dive into greater detail about each box.

Key Points

• The Copilot stack has three boxes for front end, mid-tier, and back end.

• Subsequent talks will provide more details about each box.

User Experience Design with Copilot

The speaker discusses how user experience design with Copilot is different from traditional user experience design because natural language is used instead of mapping user interface elements to code chunks.

Key Points

• Traditional user experience design involves mapping UI elements to code chunks.

• With Copilot, natural language is used instead.

• Spend less time thinking about UI widgets and more time thinking about what you want the copilot to be capable of.

Fiddling Around with User Interface Elements in Copilot

The speaker discusses how fiddling around with user interface elements is different in Copilot and emphasizes the importance of thinking about what you want the copilot to be capable of.

Key Points

• Fiddling around with UI elements is less important in Copilot.

• Think about what you want the copilot to be capable of.

• Augment models with plugins and fine-tuning.

Restraint and Safety in Copilot

The speaker discusses the importance of restraint and safety when designing a copilot, as well as the need to restrain foundation models to a particular domain.

Key Points

• Restraint and safety are important when designing a copilot.

• Foundation models are a big bucket of unrestrained capability.

• Restrain foundation models to your particular domain.

Orchestration Layer in Copilot

The speaker introduces orchestration layer in Copilot, which is responsible for sequencing through all of the models, filtering, prompt augmentation, etc. They also mention that Microsoft uses Semantic Kernel as their orchestration mechanism but other options are available.

Key Points

• Orchestration layer is responsible for sequencing through all of the models, filtering, prompt augmentation, etc.

• Microsoft uses Semantic Kernel as their orchestration mechanism.

• Other options are available.

Orchestration Layer

In this section, the speaker talks about the prompt and how it is manipulated in the orchestration layer. The speaker also discusses prompt and response filtering.

Prompt Manipulation

• A prompt is a bucket of tokens generated by the user experience layer of an application.

• The prompt can be a direct natural language thing from the user or something conveyed to the model from the application.

• Prompt filtering is used to prevent prompts that may cause unsafe responses or do not meet the needs of an application. Responses are also filtered on their way back up.

Meta Prompt

• The meta prompt is a unit of prompt code that tells the model how to accommodate itself to build a copilot. It's where safety tuning happens, and it's where you tell the model what personality you want it to have.

• Meta prompts are used for fine-tuning and making things easier than going down to lower layers in infrastructure.

Grounding

• Grounding involves adding additional contexts to prompts that may be useful for helping models respond better.

• Retrieval-augmented generation involves looking at user queries and issuing a query to search indexes for relevant documents for extra context.

• Vector databases are increasingly being used for retrieval augmented generation by computing embeddings and doing lookups indexed by those embeddings.

Plugin Execution

• Plugin execution happens when plugins are used to add extra context before prompts go down to models or when plugin execution occurs on their way back up from the model to take action on a system.

Foundation Models

• Foundation models and infrastructure are at the bottom of the stack. Choices for using foundation models in Copilot platform include using hosted foundation models, fine-tuning APIs, or training your own model from scratch.

Building a Social Media Copilot with AI

In this section, Kevin Scott talks about how he built a social media copilot using AI to advertise his podcast on social media. He explains the step-by-step process of building the copilot and emphasizes the importance of AI safety.

Building a Social Media Copilot

• The copilot runs on a Windows PC and uses open-source models and hosted models for retrieval augmented generation.

• The first step is to get a transcript from an audio file using the OpenAI Whisper model.

• Databricks Dolly 2.0, a large language model running on their Windows PC, is used to extract information from the transcript such as who was the guest in this episode.

• Information extracted from the transcript is sent to Bing API to get Neil's name and bio which are combined together into a single packet of information for social media blurb.

• Hosted OpenAI API is called to get an image from DALL-E model for thumbnail.

• A plugin for LinkedIn is invoked that takes thumbnail, post, and link to podcast and posts it on Kevin's LinkedIn feed after reviewing it.

Importance of AI Safety

• Kevin emphasizes that they think about AI safety at every step of building copilots.

• Microsoft provides amazing tools for building safe, responsible AI applications including media provenance tools that help users understand when they're seeing generated content or not.

• Cryptographic provenance watermarks can be added to AI applications that generate synthetic content using Microsoft's APIs.

• Kevin encourages developers to build interesting copilots on top of the platform and shares an anecdote about his time as an intern at Microsoft Research in 2001.

Murray's Impact on the Industry

In this section, the speaker talks about Murray and his contribution to the industry.

Murray's Contribution

• Murray figured out protected mode and got Microsoft software to work beyond that 64K memory barrier.

• The impact of small things like that had on the trajectory of the industry is unbelievable.

Inspiration from Murray

• The speaker was in awe of Murray and wondered what he could do in his career that would allow someone younger to look at him with admiration.

• The challenge for everyone now is to use new tools and capabilities to do legendary things that others will be in awe of one day.

Introduction of Executive Vice President of Cloud and AI

In this section, the speaker introduces their colleague, Executive Vice President of Cloud and AI.

Introduction

• The speaker introduces their colleague as Executive Vice President of Cloud and AI.