AWS re:Invent 2023 - Keynote with Dr. Swami Sivasubramanian

Welcome and Introduction

The Vice President of Data and AI at AWS, Dr. Swami Sivasubramanian, welcomes the audience to re:Invent 2023 and shares the vision for this year's event.

Introduction to Generative AI

• Generative AI is augmenting human productivity and fueling human intelligence and creativity.

• The relationship between humans and AI in generating new innovations is similar to symbiotic relationships observed in nature.

• Examples of symbiotic relationships include whale sharks and remora fish, as well as symbiotic stars that share gases and heat.

• Generative AI has the potential to reinvent the way we work and form new innovations.

Historical Perspective on Humans and Technology

Dr. Swami Sivasubramanian discusses the historical exploration of the symbiotic relationship between humans and technology.

Invention of New Technologies

• Over the last 200 years, visionaries like Ada Lovelace have dedicated their lives to inventing new technologies that reduce manual labor and automate complex tasks.

• Ada Lovelace recognized that computers could go beyond number crunching and perform complex tasks through logical operations.

• She speculated that computers could understand musical notation patterns and even create music.

Ada Lovelace's Contributions

Dr. Swami Sivasubramanian highlights Ada Lovelace's contributions to computing.

Shift Towards Complex Tasks

• Ada Lovelace's discovery marked a shift towards using computers for complex tasks beyond simple calculations.

• She believed that true creativity and intelligence originate from humans, while computers can only generate outputs or perform tasks ordered by humans.

• Her analysis of computing is inspiring because it recognizes the unique strengths of both humans and computers.

Co-evolution of Humans and Technology

Dr. Swami Sivasubramanian discusses the co-evolution of humans and technology, emphasizing the importance of their mutually beneficial relationship.

Inspiring Perspective

• The co-evolution of humans and technology is inspiring because it acknowledges that true creativity and intelligence originate from humans.

• This perspective is personally meaningful to Dr. Swami Sivasubramanian, as his daughter admires Ada Lovelace's contributions.

• The contributions of Ada Lovelace are gaining relevance as the world of generative AI unfolds.

Relationship Between Data, Generative AI, and Humans

Dr. Swami Sivasubramanian explores the symbiotic relationship between data, generative AI, and humans.

Foundational Models and Data

• The explosion of data has enabled the existence of foundational models that power generative AI.

• Access to a variety of language models (LMs) and foundational models (FMs) is essential for building generative AI applications.

• AWS provides a comprehensive set of AI and ML data and compute stack to train these models cost-effectively.

Three-Layer Stack for GenAI Applications

Dr. Swami Sivasubramanian explains the three-layer stack for developing GenAI applications on AWS.

Infrastructure Tier

• The infrastructure tier includes hardware chips, GPUs, and Amazon SageMaker for building, training, and deploying ML models.

• AWS offers a comprehensive set of AI and ML data and compute stack for running data-intensive and ML-intensive applications.

Middle Layer: Amazon Bedrock

• Amazon Bedrock provides access to leading LMs and other FMs for building scalable generative AI applications.

Top Layer: Applications

• The top layer includes applications like Amazon Q, an AI-powered assistant tailored to businesses, that allows users to take advantage of GenAI without specialized knowledge or coding.

Conclusion

In this transcript, Dr. Swami Sivasubramanian discusses the symbiotic relationship between humans and technology, particularly in the context of generative AI. He highlights the historical contributions of Ada Lovelace and emphasizes the co-evolution of humans and technology. The transcript also explores the importance of data in powering generative AI applications and provides an overview of AWS's three-layer stack for developing GenAI applications.

Reasons Customers Gravitate Towards Bedrock

This section discusses the reasons why customers choose Bedrock and highlights the importance of customer choice in selecting foundational models that support their unique needs.

Customer Choice and Model Evolution

• Bedrock offers a wide range of leading foundational models to cater to customers' unique needs.

• The ability to select from different models is crucial because no single model can dominate the world.

• As GenAI is still in its early days, these models will continue to evolve at unprecedented speeds.

• Customers need flexibility to use different models for different use cases, and Bedrock provides this flexibility.

Popular Foundational Models

• Anthropics Claude model is popular for tasks like summarization and complex reasoning.

• Stability AI Stable Diffusion model is used for generating images, arts, and design.

• Cohere Command is suitable for tasks like copyright and dialog, while Cohere Embed is used for search and personalization.

New Models Added

• Three new Cohere models have been added: Command Line Embeddings, Embed Multilingual, and Meta Llama 2.

• Meta Llama 2 offers high performance at a relatively low cost and has gained rapid adoption among customers.

• LLAMA 2 13 B model is optimized for various small-range use cases.

• Stable Diffusion XL 1.2 Stability AI Advance Text Image model has been added as well.

Announcement of Bedrock Support for Anthropics Claude 2.1

This section announces the support of Anthropics Claude 2.1 on Bedrock, highlighting its advancements in key capabilities for enterprises.

Advancements in Anthropics Claude 2.1

• Anthropics Claude 2.1 introduces industry-leading advancements such as a 200K context window, improving accuracy and reducing hallucinations by 50%.

• It also offers a two-fold reduction in false statements and open-ended conversations.

• Improved system prompts provide a better user experience while reducing costs.

Support for Llama 2.70B and Text Models

This section announces the support for Llama 2.70B on Bedrock and discusses the availability of text models for various use cases.

Llama 2.70B Support

• Llama 2.70B is suitable for large-scale text-based processing, including language modeling, text generation, and dialog systems.

Text Models

• Titan Text Lite is a small but cost-effective model that supports use cases like chatbot, Q&A, and text summarization.

• Titan Text Express offers optimized accuracy, performance, and cost depending on specific use cases.

Enhancing Search and Personalization with Vector Embeddings

This section discusses the use of vector embeddings to enhance search and personalization experiences for customers.

Vector Embeddings

• Vector embeddings are numerical representations produced by foundational models to translate text inputs into numerical formats suitable for machine learning.

• They enable models to find relationships between similar words, resulting in more relevant responses to customer queries.

• Vector embeddings greatly enhance the accuracy of applications such as rich media, search, and product recommendation.

Introducing Titan Multimodal Embeddings

This section introduces Titan Multimodal Embeddings, which enables the creation of richer multimodal search and recommendation options.

Titan Multimodal Embeddings

• Titan Multimodal Embeddings allow the generation, storage, retrieval of embeddings to build more accurate and contextually relevant multimodal search.

• Companies like Offer Up and Schmidt are already using Titan Multimodal Embeddings to enhance their search experiences.

Availability of Text Models

This section highlights the availability of text models, including Titan Text Lite and Titan Text Express, for optimizing accuracy, performance, and cost based on specific use cases.

Text Models for Various Use Cases

• Titan Text Lite is a small but cost-effective model suitable for chatbot, Q&A, and text summarization.

• Titan Text Express offers optimized accuracy, performance, and cost depending on specific use cases.

Introduction to Titan Image Generator

The speaker introduces the Titan Image Generator, a model that allows customers to produce high-quality, realistic images or enhance existing images using simple natural language prompts. The model is trained on diverse datasets and includes built-in mitigations for toxicity and bias through human evaluation. All Titan-generated images come with an invisible watermark designed to reduce the spread of misinformation.

• Titan Image Generator enables customers to produce high-quality, realistic images or enhance existing images.

• Customers can customize these images using their own data to better reflect their industry or brand.

• The model is trained on diverse datasets and provides more accurate outputs compared to other leading models.

• Built-in invisible watermarks are integrated into image outputs to help identify AI-generated images and reduce misinformation.

Responsible Development of AI Technology

The speaker discusses the responsible development of AI technology and how AWS is committed to promoting it. They highlight that all Titan-generated images come with an invisible watermark as a discreet mechanism to identify AI-generated images. AWS is among the first model providers to widely release built-in invisible watermarks that are tamper-resistant.

• AWS is committed to promoting the responsible development of AI technology.

• All Titan-generated images have an invisible watermark designed to help reduce the spread of misinformation.

• The watermark serves as a discreet mechanism for identifying AI-generated images.

• AWS is one of the first providers to release tamper-resistant built-in invisible watermarks.

Demonstration of Model Editing Features

The speaker demonstrates some editing features of the Titan Image Generator. They use a text prompt, such as "image of a green iguana," and show how they can easily swap out backgrounds, generate variations of the original subject and background, and change the orientation of the picture.

• The speaker uses the Titan Image Generator to swap out backgrounds and generate variations of an image.

• They demonstrate output painting, which involves swapping out existing backgrounds with new ones.

• The model allows for seamless background swaps while retaining the main subject of the image.

• The orientation of the picture can also be changed using prompts, such as "left-facing to right-facing."

Broad Range of Domains for Customers

The speaker mentions that customers across a variety of industries will be excited to take advantage of the Titan Image Generator. Each Titan model has its own unique strengths in terms of capabilities, price, and performance. AWS carefully chooses how they train their models and the data they use.

• Customers from various industries will benefit from using the Titan Image Generator.

• Each Titan model has unique strengths in terms of capabilities, price, and performance.

• AWS is selective in training their models and choosing relevant data.

Customer Use Cases

The speaker shares examples of customer use cases for Bedrock-powered applications. These include self-service, customer support, text analysis, forecasting trends, automation within SAP Concur for trip requests, chat assistants for retrieving critical factory data at Georgia-Pacific, and accessing up-to-date information at US and United Airlines.

• Over 10,000 customers have rapidly developed GenAI-powered applications since Bedrock's launch.

• Use cases include self-service, customer support, text analysis, forecasting trends.

• SAP Concur automates trip requests using Bedrock.

• Georgia-Pacific uses a chat assistant powered by Bedrock to retrieve critical factory data.

• US and United Airlines utilize Bedrock to help employees access up-to-date information on delays.

Putting GenAI into Action for Your Business

The speaker introduces Nhung Ho, VP of AI from Intuit, who will demonstrate how to put GenAI into action for businesses. They highlight Intuit's mission to power prosperity for 100 million consumers and small business customers and their transformation journey over the past five years.

• Nhung Ho from Intuit will demonstrate how to put GenAI into action.

• Intuit's mission is to power prosperity for consumers and small business customers.

• They have undergone a transformation journey over the past five years.

• AWS, including SageMaker, plays a foundational role in enabling their machine learning capabilities.

Empowering Small Businesses with AI

Nhung Ho shares her personal connection to the challenges faced by small business owners as she comes from a large family of small business owners. She emphasizes that at Intuit, they aim to make their lives easier through AI and build applications that solve problems for their customers.

• Nhung Ho has a personal understanding of the challenges faced by small business owners.

• At Intuit, they focus on making their lives easier through AI-powered applications.

• Their goal is to level the playing field for their customers.

Partnership with AWS and Scale Achievements

Nhung Ho discusses Intuit's partnership with AWS and how it has enabled them to achieve incredible scale. They run all their data capabilities and data lake on AWS, while SageMaker serves as a foundational capability for their machine learning platform.

• Intuit has partnered with AWS for running data capabilities and data lake.

• SageMaker is a foundational capability in their machine learning platform.

• The partnership has allowed them to achieve incredible scale.

Due to limitations in the provided transcript, the summary may not capture all the details and nuances of the video.

Interactions per Year

In the era of generative AI, the speaker discusses how their investment in clean data, strong data governance, and responsible AI principles has allowed them to unlock new opportunities for designing and shipping AI applications.

Generative AI and Investment in Data

• The company has made a multiyear investment in ensuring clean data, strong data governance, and responsible AI principles.

• This investment has positioned them well to change the game in the era of generative AI.

• They have been able to quickly unlock new opportunities for technologists to design, build, and ship out AI applications.

GenAI Operating System - Genomics

• The company has built a proprietary GenAI operating system called Genomics on AWS.

• Genomics consists of four primary components.

• Gen Studio: Allows anyone at the company to build, prototype, and test new GenAI experiences.

• Gen Runtime: Provides connectivity to multiple learning management systems (LMS) and access to underlying data for building personalized and accurate experiences that can be scaled when needed.

• Design System (Gen X): Ensures consistency across products while providing a seamless customer experience when interacting with GenAI.

• Financial Large Language Models (LLMs): A set of third-party LLMs as well as custom-trained LLMs specialized in tax accounting, marketing, and personal finance.

Importance of Data, Accuracy, Latency, and Cost

• Data is key to unlocking accuracy in building AI experiences.

• Using smaller and faster models allows for significant latency gains.

• Hosting models on SageMaker helps manage cost effectively according to needs.

• Third-party LLMs are used because optimizing for the best customer experience requires using best-in-class solutions.

Intuit Assist - Generative AI Assistant

The speaker introduces Intuit Assist, their generative AI assistant embedded across all product offerings. They aim to help customers feel confident in every financial decision they make.

TurboTax Example

• Intuit Assist is live in production for customers using TurboTax and other products like QuickBooks and Mailchimp.

• Customer feedback has been gathered to improve the experience.

• Intuit Assist helps users understand the outcome of their tax filing experience by unpacking the numbers and providing a clear explanation.

• The combination of knowledge engine and LLMs ensures accuracy and helps users feel confident in their financial decisions.

Advice for AI Journeys

• Take a holistic approach and invest in underlying data to differentiate each experience built.

• Build horizontal solutions from day one to smoothly transition from demos to production experiences.

• There is no one-size-fits-all LLM solution; optionality is important, which is offered on Bedrock (AWS platform).

Transformations at Intuit

The speaker highlights that GenAI is just one of many transformations at Intuit over its 40-year history. They emphasize the importance of data as a differentiator for creating unique GenAI applications.

Importance of Data for GenAI Applications

• Data is the key differentiator between generic AI applications and GenAI applications that understand customers' specific needs.

• Building GenAI applications requires leveraging customer data to create personalized experiences.

Conclusion

The speaker concludes by emphasizing the critical role of data in creating unique GenAI applications tailored to businesses. They mention their collaboration with AWS over the past ten years, which has helped them become a global financial technology platform.

Critical Role of Data

• Your data is crucial for building GenAI applications that are unique to your business.

• Data differentiates a generic AI application from a GenAI application that understands customers and their specific needs.

Collaboration with AWS

• Intuit's collaboration with AWS over the past ten years has helped them become a global financial technology platform.

• They have utilized various services offered by AWS to support their growth and transformation.

The transcript provided is in English, so the response is also in English.

Knowledge Customization and Fine-Tuning with Amazon Bedrock

This section discusses the importance of knowledge customization and fine-tuning in your industry. It introduces Amazon Bedrock as a tool that removes heavy lifting from the fine-tuning process and allows leveraging unlabeled datasets or raw data to maintain accuracy. Pre-training processes are also mentioned, such as using medical journals or research papers to make the model more knowledgeable on evolving industry terminology.

Knowledge Customization with Amazon Titan Lite and Titan Express

• Amazon Titan Lite and Titan Express are models that complement each other and enable understanding of your business over time.

• The output model is accessible only to you and never goes back to the base model.

Retrieval Augmented Generation (RAG) for Up-to-Date Information

• Retrieval Augmented Generation (RAG) is a technique used when data changes frequently, such as inventory or pricing.

• RAG helps enable a model with up-to-date information by augmenting prompts with contextual information drawn from private data sources.

• Implementing RAG-based systems can be complex, involving converting data into vector embeddings, storing them in a vector database, and building custom integrations.

Knowledge Bases for Amazon Bedrock

• Yesterday, knowledge bases were announced for Amazon Bedrock, supporting the entire RAG workflow from ingestion to retrieval to prompt augmentation.

• Knowledge bases simplify the process by pointing to the location of your data (e.g., an S3 bucket), fetching relevant context and text documents, converting them into embeddings, storing them in your vector database, and augmenting prompts during inference.

• Popular vector databases like Vector Engine for OpenSearch Serverless Redis Enterprise Cloud and Pinecone are supported.

Agents for Amazon Bedrock: Executing Business Tasks

This section introduces agents for Amazon Bedrock as a capability that enables GenAI applications to execute complex tasks by dynamically invoking APIs. It simplifies resource-intensive steps like defining instructions and orchestrating models to access data sources.

Customizing Models with Your Data

• Developers can customize models with their data to execute business tasks like booking travel or processing insurance claims.

• Resource-intensive steps are involved in fulfilling user requests, such as defining instructions and configuring models to access data sources.

Agents for Amazon Bedrock

• Yesterday, the general availability (GA) of agents for Amazon Bedrock was announced.

• Agents enable the creation of fully managed agents that connect to internal systems and APIs on behalf of developers in just a few steps.

Hypothetical Scenario: Leveraging GenAI Capabilities

This section presents a hypothetical scenario of leveraging GenAI capabilities for a DIY project using an app called RAD DIY powered by a GenAI-powered assistant with Claude 2 on Amazon Bedrock.

RAD DIY App for DIY Projects

• The RAD DIY app is designed to remove complexities from DIY projects and provide customers with accurate and easy-to-follow steps.

• Users can ask the assistant about any type of project using natural language and receive detailed steps, materials, tools, permits, and even images generated by the Titan Image Generator model.

• The app uses multi-modal embeddings to search its external inventory and retrieve all necessary products without multiple trips to the store.

• Cohere Command model provides summaries of user reviews for each reviewed product, helping users make informed decisions.

Conclusion

The conclusion summarizes the importance of knowledge customization, fine-tuning, retrieval augmented generation (RAG), agents for Amazon Bedrock, and leveraging GenAI capabilities in various industries. It highlights how these tools simplify complex tasks and enhance customer experiences.

Building GENAI Applications with Bedrock

This section discusses how to use Bedrock to build GENAI applications and create new customer experiences. It introduces the AWS GenAI Innovation Center program for hands-on support and customization of foundational models.

Introduction to Bedrock and GENAI

• Hypothetical scenario sparks ideas on using Bedrock to build GENAI applications with data for new customer experiences.

AWS GenAI Innovation Center Program

• AWS offers the GenAI Innovation Center program for hands-on support in getting started with GENAI.

• The program pairs customer teams with AI/ML experts to access their GENAI journey.

• Customers can customize foundational models through the Innovation Center's custom model program.

Customizing Foundational Models

• The new Innovation Center Custom Model Program allows customization of Anthropoclaude models for business needs using proprietary data.

• Scoping requirements, defining evaluation criteria, and working with proprietary data are part of the customization process.

• Fine-tuned private models can be securely accessed and deployed on Bedrock within a VPC.

Alternative Approach - Building Own AI Applications

• Some companies may need to build their own AI applications and require powerful machine learning infrastructure.

• AWS has partnered with NVIDIA for high-performance GPU solutions for deep learning workloads.

• AWS has also invested in ML chips like AWS Trainium and AWS AI Inferentia for cost efficiency and performance.

Amazon SageMaker for ML Model Development

• Amazon SageMaker provides best-in-class software tools for building, training, and deploying ML models.

• Efficient model training is supported with distributed training libraries and built-in tools to improve performance.

• Leading organizations like Stability AI, AI 21 Labs, Hugging Face, and Tay AI are using Amazon SageMaker for training foundational models.

Challenges in Training Foundational Models

• Training foundational models can be challenging due to the massive size of the models and data sets used.

• Developers need to split data into chunks and distribute them across a cluster of accelerators.

• Writing code, optimizing it frequently, pausing to inspect model performance, and manually remediating hardware issues are additional challenges.

Introducing SageMaker HyperPods

• SageMaker HyperPods are a new distributed training capability that reduces model training time by up to 40%.

• Pre-configured with distributed training libraries, HyperPods efficiently distribute data and models across thousands of chips in a cluster.

• HyperPods automatically take checkpoints frequently, allowing iterative inspection and optimization of models. It replaces faulty instances if hardware failures occur during training.

Other SageMaker Innovations

• Several new features and capabilities have been introduced for SageMaker, including inference improvements, MLOps enhancements, and updates to SageMaker Studio.

• These innovations make it easier for customers to build, train, and deploy large language models.

Leveraging SageMaker Innovations - Perplexity's Experience

This section introduces Perplexity, a customer leveraging the latest SageMaker innovations for training and deploying their own models.

Introduction to Perplexity

• Perplexity aims to be the world's leading conversational answer engine that directly answers questions with references.

Leveraging SageMaker Innovations

• Jens from Perplexity is training and deploying their own models using the latest SageMaker innovations.

Perplexity's Generative User Interfaces on AWS

This section discusses Perplexity's generative user interfaces and their deployment on AWS. They highlight the complexity of the product and the decision to use AWS for testing and deploying their models.

Testing Frontier Models on AWS Bedrock

• Perplexity tested frontier models like Anthropi, Claude 2 on AWS Bedrock.

• Bedrock provided cutting-edge inference for these frontier models.

• The testing helped improve general question answering capabilities by providing more natural-sounding answers.

• Claude 2 also introduced new capabilities into the product, such as uploading multiple large files and asking questions about their contents.

Orchestrating Multiple Models in One Product

• Perplexity orchestrates several different models in one single product, including those they have trained themselves.

• They built on top of open-source models like Llama 2 and Mistral, fine-tuning them for accuracy and grounding them with web search data using cutting-edge rack technology.

Collaboration with Amazon Sagemaker Hyper Bot

• Perplexity worked with the startups team at Amazon Sagemaker to develop a proof-of-concept (POC) called Sagemaker Hyper Bot.

• Sagemaker Hyper Bot made it easier to debug large model training and handle distributed capacity efficiently.

• They obtained AWS EC2 P4 capacity for training, enabling them to fine-tune state-of-the-art open-source models like Llama 2 and Maestro.

Benefits of AWS for Training and Inference

• Moving to Hyper Pod and enabling AWS Elastic Fabric Adapter resulted in a significant increase in training throughput.

• AWS also provided customized services to support Perplexity's inferencing needs, especially on P4 and P5 instances.

• These benefits allowed Perplexity to build top-of-the-market APIs for open-source models and their in-house models.

General Availability of Models as APIs

• Perplexity is excited to announce the general availability of their models in the form of APIs.

• These APIs include live APIs that have no knowledge cut and are plugged into their search index, all fully hosted on AWS.

The Future of Generative AI with Perplexity

This section highlights Perplexity's vision for generative AI and how they aim to disrupt and innovate the search experience. They express gratitude for working with AWS and emphasize the importance of data foundation for genAI applications.

A Glorious Revolution for Consumers

• Perplexity believes that generative AI is still in its nascent stages and foresees a glorious revolution where consumers will benefit from great new product experiences and competitive pricing.

• They strive to close the research-to-decision-to-action loop further, making genAI technology a seamless part of users' lives in the years to come.

Disruption and Innovation at Its Prime

• Perplexity aims to be the Earth's most knowledge-centric company.

• They express gratitude for working with AWS, ensuring that users never have to go back to traditional search engines like Ten Blue Link Search Engine.

Building a Strong Data Foundation for GenAI

This section emphasizes the importance of a strong data foundation when building genAI applications. It discusses the need for comprehensive services, tools, and governance across the end-to-end data workflow.

Comprehensive Set of Services

• Accessing a comprehensive set of services is crucial when dealing with scale, volume, and various types of use cases.

• AWS offers a broad range of tools for storing, organizing, and accessing different types of data, including relational databases like Amazon Aurora and Amazon RDS, non-relational databases like Amazon DynamoDB, analytics tools like Amazon Redshift, etc.

Tools for Acting on Data

• In addition to ML and genAI tools, there is a need for services that deliver insights from data.

• Amazon QuickSight is mentioned as a unified BI service that helps deliver insights from data.

Cataloging and Governing Data

• It is important to have services that help centralize access controls and govern data across various areas.

• AWS provides the right tools for storing, retrieving, indexing, and searching vector embeddings.

Adding Vector Capabilities to Popular Data Sources

This section highlights AWS's investment in adding vector capabilities to popular data sources like Amazon Aurora, Amazon RDS, OpenSearch Service, etc., based on customer feedback and demand.

Benefits of Vector Capabilities

• Customers expressed their desire to use vectors in their existing databases to eliminate the learning curve associated with new programming paradigms.

• By adding vector capabilities to familiar databases, customers feel more confident about scalability and availability while meeting the needs of vector databases.

• Storing vectors and business data in the same place improves application performance without worrying about data movement or synchronization.

Conclusion

Perplexity's generative user interfaces deployed on AWS showcase their commitment to innovation in search technology. They leverage AWS services like Bedrock, Sagemaker Hyper Bot, EC2 P4 capacity, Elastic Fabric Adapter, etc., to improve question answering capabilities and provide top-of-the-market APIs. Perplexity envisions a future where generative AI revolutionizes consumer experiences. Building a strong data foundation is crucial for genAI applications, which can be achieved through comprehensive services and tools provided by AWS. The addition of vector capabilities to popular data sources further enhances the usability of genAI technologies.

Performance Improvements and Vector Capabilities

In this section, the speaker discusses the performance improvements made to existing services and introduces vector capabilities for popular databases.

Aurora Optimized Reads

• Aurora Optimized Reads now support billions of vectors with a 20x improvement in queries per second performance.

• Single-digit millisecond latency has been achieved.

• Ongoing investments are being made to further improve performance in these areas.

Vector Capabilities for DocumentDB and DynamoDB

• Vector capabilities are introduced for DocumentDB and DynamoDB, two popular databases.

• Customers using DocumentDB can now store their source data and vector data together in the same database.

• DynamoDB customers can access vector capabilities through a zero-ETL integration with Amazon OpenSearch.

Introduction of Amazon MemoryDB for Redis

• Amazon MemoryDB for Redis is a purpose-built data store that provides ultra-fast performance for in-memory vector databases.

• Customers can achieve millisecond response time even at high recall and throughput.

Preview of Vector Search in MemoryDB

• Vector Search is now available in preview for MemoryDB for Redis.

• Customers can benefit from ultra-fast vector search with high throughput and concurrency.

• Millions of vectors can be stored, providing single-digit millisecond response time even under heavy query loads.

Combining Graph Analytics with Vector Search

• Neptune Analytics, an analytics engine for Neptune (a graph database), is announced as generally available.

• Data scientists and app developers can analyze large amounts of graph data up to 80x faster using Neptune Analytics.

• The combination of graph analytics and vector search allows uncovering hidden relationships across data quickly.

Integrated Data Foundation

This section focuses on integrating data across different sources to create a complete view of the business.

Zero ETL Integration Efforts

• Amazon has invested in building seamless integrations across its data stores to enable a zero-ETL future.

• Fully managed zero-ETL integration is available between Aurora, MySQL, and Redshift for near real-time analytics.

• Additional zero-ETL integrations have been announced, including Aurora PostgreSQL, RDS for MySQL, and DynamoDB with Redshift.

Zero ETL Integration Between Amazon OpenSearch and S3

• A new zero ETL integration is introduced between Amazon OpenSearch and S3.

• This integration enables seamless search, analysis, and visualization of all log data in one place without creating ETL pipelines.

Benefits of Integrated Data Foundation

• Breaking down data silos across databases, data lakes, data warehouses, and third-party sources allows for better customer experiences.

• Integrated data foundation ensures that data is readily accessible for GenAI apps.

• Building and managing ETL pipelines has been a traditional pain point for customers.

Conclusion

The speaker discussed the performance improvements made to existing services such as Aurora Optimized Reads. Vector capabilities were introduced for popular databases like DocumentDB and DynamoDB. The addition of Amazon MemoryDB for Redis provides ultra-fast performance for in-memory vector databases. The combination of graph analytics with vector search was highlighted through Neptune Analytics. The importance of an integrated data foundation was emphasized to create a complete view of the business by breaking down data silos and enabling zero ETL integrations.

The Overhead for Managing Multiple Tools

This section discusses the challenges of managing multiple tools and introduces a solution for performing complex queries and forensic analysis across multiple sources.

Challenges of Managing Multiple Tools

• Organizations face difficulties in managing multiple tools.

• Complex queries for forensic analysis and data correlation across multiple sources are required.

• Zero-ETL integration helps protect against service downtime and security events.

Introduction to Data Foundation

• Amazon DataZone is a data management service that helps catalog, discover, share, and govern data within an organization.

• It enables employees to collaborate with data and drive insights for the business.

• DataZone is used by companies like Gartner Health to focus on building cancer solutions instead of worrying about building a governance platform.

Securely Sharing Data with AWS

• Clean Rooms allow customers to securely share data with their selected partner, AWS.

• Clean Rooms enable safe analysis of collected data and generation of insights without sharing the entire dataset.

• Customers want to run machine learning models on clean rooms to get predictive insights.

Introducing Clean Rooms ML

• Clean Rooms ML allows customers to train private lookalike models across collective data without sharing the underlying data.

• It saves development time and resources by easily applying ML models in just a few steps.

• Lookalike modeling is available now, with more models planned for introduction in the future.

Applications Powered by GenAI at Booking.com

This section focuses on how Booking.com utilizes generative AI technology to enhance customer experiences.

Introduction to Booking.com

• Booking.com is a two-sided marketplace offering accommodations, flights, rental cars, attractions globally.

• They manage over 28 million listings of places to stay and provide services in 54 countries.

• Partnering with AWS helped Booking.com tackle data challenges and improve their data strategy.

Leveraging Generative AI

• Booking.com built the AI Trip Planner using generative AI technology.

• The AI Trip Planner allows users to book trips in a conversational manner.

• They implemented the intent detection model LLAMA 2 hosted on Amazon SageMaker.

Reflections on Technology and Innovations at Booking.com

This section reflects on the evolution of technology and highlights the use of generative AI for innovations at Booking.com.

Early Attempts at Human-like Interaction with Technology

• Reflecting on the arc of technology, early attempts like ELIZA were frustrating in providing human-like interaction.

• With the emergence of generative AI, Booking.com developed the AI Trip Planner for easier trip booking conversations.

Scale and Data Challenges at Booking.com

• Booking.com manages over 150PB of data, including millions of listings, flights, rental cars, and attractions globally.

• Partnering with AWS improved data scientists' productivity by increasing concurrent job training capacity and reducing job failures.

Building the AI Trip Planner

• Open-source model LLAMA 2 was chosen for implementing an intent detection model in the AI Trip Planner.

• Conversational interactions are enabled through a conversational interface powered by generative AI technology.

The transcript provided does not cover all sections of the video.

Protecting Customer Privacy and Leveraging Reviews

In this section, the speaker discusses the importance of protecting customer privacy and leveraging reviews to enhance the booking experience.

Importance of Customer Privacy

• Customers tend to input their personally identifiable information in the trip planner.

• The company prioritizes protecting customer privacy by stripping out such information.

Leveraging Reviews for Decision Making

• Customers highly value reviews when making travel decisions.

• The company's implementation leverages AWS technology to pull in review data.

• Machine learning algorithms help create personalized hotel recommendation options for travelers.

• A JSON object is populated to communicate with the recommendation engine.

• The app provides a carousel for users to easily browse and book hotels.

Collaboration with Bedrock and Titan Teams

This section highlights the collaboration between Booking.com and Bedrock teams, as well as their work with the Titan teams.

Collaborating with Bedrock Teams

• Booking.com is already making use of Sagemaker technology.

• Close collaboration with Bedrock teams on upcoming projects.

Collaboration with Titan Teams

• The speaker suggests covering the entire ARC (Amazon Redshift Console).

• Seeking input from Titan teams on what should be discussed.

Introduction to Generative AI for Personalized Booking Experience

The speaker introduces generative AI as a new technology that can personalize the booking experience based on vast amounts of booking data.

Personalized Hotel Recommendations using GenAI

• GenAI is used at Booking.com to create personalized hotel recommendations tailored to each user's unique needs and preferences.

• Users are encouraged to visit Booking.com for a customized booking experience.

Enhancing Data Foundation with ML and AI

This section focuses on how ML and AI can enhance data management and analytics.

ML and AI as Data Management Tools

• ML and AI can transform the way data is managed.

• Booking.com has infused AI and ML across many of its data services to remove heavy lifting from data management and analytics.

• Customers are interested in leapfrogging their data strategy with GenAI.

Optimizing Data Warehouses with AI

This section discusses how AI can optimize the performance of data warehouses.

Challenges in Data Warehouse Management

• Managing data variability, concurrent users, query complexity, price, and performance presents challenges.

• Multi-variant optimizations are difficult for humans but excel with machine learning algorithms.

AI-driven Scaling and Optimizations for Redshift Serverless

• Redshift Serverless offers proactive scaling on multiple dimensions simultaneously.

• Customers benefit from faster queries with optimal price-performance tradeoff.

Supporting Data Management with AI

This section explores how AI can support data management beyond optimizing data warehouses.

Honda's Use Case: Better Price Performance and Actionable Insights

• Honda leverages AI-driven scaling and optimizations to deliver better price performance and gain insights from vehicle data points without manual intervention.

Amazon MQ for Data Management Support

• Amazon MQ is a GenAI-powered assistant tailored to businesses.

• It connects to your data, provides contextual information based on your role, internal processes, governance policies, and removes heavy lifting from repetitive tasks like running data queries.

Leveraging Amazon Q for Querying Data Warehouse

This section highlights the use of Amazon Q to support coding language for querying the data warehouse.

Simplifying SQL Queries with Amazon Q

• Developers spend a significant portion of their time on tasks like looking up documentation or building/testing new features.

• Amazon Q leverages GenAI's natural language translation capabilities to support coding language for daily data warehouse queries.

Tools for SQL Builders

• Redshift Query Editor offers features like autocomplete suggestions, syntax validation, and detailed detections during table creation to simplify SQL queries.

Conclusion

The transcript covers various topics related to protecting customer privacy, leveraging reviews, collaboration with teams, generative AI for personalized booking experiences, enhancing data foundation with ML and AI, optimizing data warehouses with AI-driven scaling and optimizations, supporting data management with AI through Amazon MQ, and simplifying SQL queries using Amazon Q.

Query Editor with Q

The Query Editor with Q allows users to ask questions in plain English and receive SQL query recommendations in seconds. It analyzes the schema and provides accurate and relevant recommendations based on the data.

Using Q for Data Analysis

• Users can ask questions in plain English, such as finding the top three venues that sold the most tickets.

• Q analyzes the schema and generates SQL query recommendations quickly.

• Users can add the recommended queries to their notebook and test them out.

Retrieving Relevant Data

• Q knows where to find information on ticket sales (in the sales table) and event details (in the event table).

• Users can ask Q to find out which event types were most popular at specific venues.

• Results can be refined based on total events instead of total tickets for more accurate insights.

Simplifying Data Management

• Q can help with data management tasks like building data pipelines.

• Zero-ETL integrations eliminate the need for complex data pipelines between different sources.

• Custom ETL jobs can be simplified and maintained easily using Ali Q.

Creating Data Integration Pipelines with Natural Language

Soon, users will be able to use Q for creating data integration pipelines using natural language. This new integration will allow faster creation of data integration jobs, troubleshooting through a chat interface, and instant integration help without prior knowledge of Amazon Glue.

Example of Data Integration Pipeline Creation

• Users can ask Q to read data from S3, drop null records, and write results to Redshift.

• Q will generate an end-to-end data integration job that performs this action seamlessly.

• The integration uses agents for Amazon Bedrock to break down prompts into specific tasks and combine results into integration jobs.

Overcoming Data Challenges for Innovation

Getting the right data and using it effectively to tell a story can be challenging. However, with the tools discussed in this session, these challenges can be overcome, leading to better decision-making and innovation.

The Challenge of Getting Data

• Obtaining the necessary data and using it in the desired format is often difficult.

• The example of rebooking flights without stress is used to illustrate the need for wrangling data from various sources.

Zero-ETL Integrations for Data Management

• Zero-ETL integrations across AWS services help bring together different data sources without building complex pipelines or writing code.

• Creating a Redshift serverless data warehouse provides a central location for all data.

• Data sharing between warehouses, integration between Aurora and Redshift, and real-time data ingestion through Redshift streaming are possible with zero-ETL integrations.

Measuring Success with Amazon Q in QuickSight

Amazon Q in QuickSight allows users to measure success by bringing key metrics and critical data together. Users can create executive summaries and build customizable data stories based on actual business data.

Creating an Executive Summary Dashboard

• Using Amazon Q in QuickSight, users can create an executive summary dashboard that highlights important insights.

• Key metrics like the time taken to rebook a flight can be visualized to showcase the impact of new features.

Building Customizable Data Stories

• With Amazon Q, users can easily transform paragraphs into bullet points for clearer takeaways.

• Data stories created with Q are fully customizable and can be securely shared within an organization to drive better decision-making.

Huron AI at the UN General Assembly Event

This section discusses how Huron AI can make a significant impact in the world by providing accessible cancer care regardless of location.

Huron AI's Mission and Impact

• Dr. Kingsley built Huron AI with the mission to ensure that the value of life does not determine whether someone lives or dies from cancer.

• In Latin America and Africa, there is a severe shortage of oncologists, with approximately one oncologist for every 3200 cancer patients.

• By comparison, the US has a ratio of about one oncologist for every 300 patients.

• Rwanda, with a population of about 13.5 million people, has fewer than 15 oncologists.

• This scarcity forces patients to travel long distances to medical facilities and often delays reporting symptoms until they become severe.

• Huron AI created applications to improve cancer care access for everyone, regardless of their location.

Advancements in Cancer Care

• Huron AI is revolutionizing cancer care detection and diagnosis predictions in Kenya, Nigeria, Ghana, and Rwanda.

• They are also filling critical gaps in cancer data for underrepresented populations, advancing equitable biopharma research and development.

GenAI Augmenting Human Abilities

This section highlights how GenAI can enhance human abilities to solve critical problems and accelerate productivity.

Amazon Codewhisperer

• Amazon Codewhisperer is an AI-powered coding companion trained on billions of lines of code.

• It provides real-time code recommendations within your integrated development environment (IDE).

• Customization features allow it to generate more relevant recommendations based on your internal codebase.

MQ: Removing Heavy Lifting from Common Tasks

• MQ helps remove the heavy lifting from common tasks across various job functions.

• It accelerates productivity in building applications, creating reports, presenting data, and working in call centers.

Toyota's Use of GenAI

This section showcases how Toyota is infusing GenAI capabilities into their products to create assistive experiences for their customers.

Safety and Data

• Toyota emphasizes the importance of data in ensuring safety.

• They use data from hundreds of sensors in vehicles globally to determine if a vehicle has been in a collision and provide assistance to emergency responders.

Amazon Bedrock and Generative AI

• Toyota utilizes newer technologies like generative AI with the help of managed services like Amazon Bedrock.

• They have developed a generative AI-powered assistant that can provide information about various aspects of the vehicle through simple voice commands.

Harnessing Human Intellect with Data and GenAI

This section discusses the relationship between data, human intellect, and GenAI in driving innovation and creating impactful customer experiences.

The Power of Human Intellect

• While technology is crucial for innovation, harnessing human intellect is equally essential.

• Creating customer experiences that make a bigger impact requires combining unstructured data with GenAI capabilities.

Example: Virola Tree in Panama Rainforest

• The example highlights how unstructured data can be used to reinforce the cycle of innovation over time.

• The Virola tree in the Panama rainforest produces small red fruits popular among local wildlife.

• By studying this tree's characteristics using unstructured data, valuable insights can be gained for future research or conservation efforts.

The Relationship Between Toucans, Agoutis, and Trees

This section discusses the relationship between toucans, agoutis, and trees in the forest ecosystem. Toucans help spread seeds by eating fruits and then dispersing the seeds through their droppings. Agoutis collect and bury these seeds for safekeeping, allowing new trees to grow when conditions are right. This cycle continues for decades or even hundreds of years.

• The toucan helps both the forest and the agouti by spreading seeds through its droppings.

• Agoutis collect and bury as many seeds as possible in the soil for safekeeping.

• These buried seeds sprout into new trees when conditions are favorable.

• This cycle of seed dispersal and tree growth continues for long periods of time.

Humans, Data, and Generative AI

This section draws a parallel between the relationship of humans, data, and generative AI with that of toucans, agoutis, and trees. It highlights how collaboration and facilitation create longevity in both scenarios.

• The relationship between humans, data, and generative AI creates longevity similar to that seen in the toucan-agouti-tree cycle.

• Collaboration and facilitation play a crucial role in strengthening each element over time.

• Humans provide unique benefits that contribute to more efficient, responsible, and differentiated generative AI applications.

• Humans are responsible for creating innovation that generates data necessary for generative AI technology.

Model Evaluation Process

This section discusses the model evaluation process in generative AI strategies. It emphasizes the importance of human feedback in selecting the best model for specific use cases.

• Human feedback is integrated into the model evaluation and selection process in generative AI strategies.

• Model evaluation requires expertise in data science and can be a tedious and time-consuming process.

• Benchmarking datasets, metrics, algorithms, and human evaluation workflows need to be created for accurate model evaluation.

• The process of model evaluation needs to be repeated periodically as new models are produced or fine-tuned.

Introducing Model Evaluation on Amazon Bedrock

This section introduces the preview of model evaluation on Amazon Bedrock, a new capability that allows users to evaluate, compare, and select the best foundational model for their use case.

• Amazon Bedrock now offers a preview of model evaluation capability.

• Users can leverage curated datasets or their own datasets for automatic evaluations.

• Qualitative and quantitative criteria such as robustness, toxicity, and accuracy can be evaluated automatically.

• Brand voice and subjective criteria can also be evaluated with human inputs through a fully managed human review workflow.

• Comprehensive reports are provided to easily review metrics on model performance.

Simplifying Model Evaluation with Amazon Bedrock

This section highlights how Amazon Bedrock simplifies the model evaluation process by providing tools for customers to evaluate models based on their specific needs.

• Customers can evaluate models for their specific needs using Amazon Bedrock's capabilities.

• Automatic evaluations can be performed using curated datasets or user-provided datasets.

• Human inputs are crucial in the development process of generative AI technology.

• As technology evolves, employers will need individuals with skills like creativity, ethics, adaptability alongside technical skills.

Reskilling Revolution with GenAI

This section discusses the importance of reskilling in preparation for the adoption of AI technologies and the emergence of new roles and products.

• The World Economic Forum predicts that around 75% of companies will adopt AI technologies by 2027.

• Reskilling is crucial to adapt to the changing landscape and unlock the potential of GenAI.

• Skills like creativity, ethics, and adaptability will become increasingly critical alongside technical skills.

• AWS is investing in reskilling programs such as the AWS GenAI Scholarship with Udacity to support students and professionals joining the reskilling revolution.

Learning Opportunities with GenAI

This section highlights various learning opportunities provided by AWS to acquire new skills in GenAI.

• AWS offers over 100 AI and ML courses and low-cost trainings for individuals to build new skills in GenAI.

• Programs like Party Rock and Amazon Bedrock Playground provide accessible ways to learn about GenAI through hands-on code-free app building.

• These tools enable users to experiment, learn, and create their own applications using powerful foundational models from leading AI companies.

Building a Party Rock App

This section demonstrates how easy it is to build a Party Rock app using Amazon Bedrock Playground.

• Users can build a Party Rock app without requiring an account or login.

• The process starts on the home page by providing a brief description of what the app should do.

• Different prompts can be experimented with, utilizing various engineering techniques for generating responses.

• Additional widgets like chatbots can be added to enhance the application's usefulness and user experience.

• Different models can be selected based on their suitability for specific use cases.

• Once satisfied with the results, users can publish their application for others to use or remix.

The Power of Data and Human Creativity

This section highlights the symbiotic relationship between data and human creativity, emphasizing the role of data in accelerating innovation and creating differentiated experiences.

Unique Inputs and Ideas

• Albert Einstein once said, "Creativity is seeing what others see and thinking what no one else thought."

• The powerful symbiotic relationship between data and humans is accelerating our ability to create new innovations.

• Data helps us see what others see, while human creativity allows us to think differently.

Accelerating Innovation

• The combination of data intelligence (GENAI) and human input is driving the acceleration of innovation.

• GENAI-powered services strengthen the foundation of data tools, allowing for customization with user-specific data.

• These tools augment employee productivity and provide mechanisms for improving processes.

• Getting human feedback is crucial in unlocking the transformative potential of this technology.

Customization with Data

• GENAI offers a secure place to customize foundational models with user-specific data.

• By leveraging GENAI-powered services, organizations can strengthen their data foundation.

• This customization enables organizations to create differentiated experiences based on their unique datasets.

Future Potential

• The transcript mentions that we are just getting started in harnessing the power of this transformative technology.

• Tools like Party Rock are available to help kick-start the utilization of these innovative capabilities.

Unlocking Transformative Technology

This section emphasizes the importance of utilizing transformative technology by providing mechanisms for customization, employee productivity enhancement, and gathering human feedback.

Strengthening Data Foundation

• GENAI-powered services offer tools designed to augment employee productivity within an organization's data foundation.

• These tools aim to improve processes by providing mechanisms for enhancing efficiency and effectiveness.

Customization with User-Specific Data

• GENAI provides a secure environment for organizations to customize their foundational models with their own data.

• This customization allows organizations to tailor their data tools and processes according to their specific needs.

Gathering Human Feedback

• The transcript mentions the importance of getting human feedback in order to fully unlock the potential of transformative technology.

• By incorporating human insights and perspectives, organizations can further enhance the capabilities and impact of these technologies.

Transformative Potential

• The transcript highlights the transformative potential of this technology, indicating that we are just scratching the surface of what it can achieve.

• Utilizing tools like Party Rock can help organizations kick-start their journey towards leveraging these transformative capabilities.