RedisDays Virtual: Supercharge Your LLM Apps With Redis and Langchain

Introduction

The speakers introduce themselves and the topic of the session.

Speakers Introduction

• Tyler Hatchison, a senior applied AI engineer at Redis, and Harrison Chase, the CEO of Langchain, are introduced as the speakers for the session.

• They will be discussing how Redis and Langchain can power LOM applications in production.

Session Overview

The speakers provide an overview of what they will be covering during the session.

Session Outline

• The session will start with a brief overview of the AI landscape.

• Harrison will then discuss the Langchain project and simplifying application development.

• Tyler will share some new and old features of Redis that lend itself well to being used in modern AI landscapes.

• Lastly, they will round it out with some example use cases and code samples.

AI Landscape Overview

The speakers provide an overview of how AI has evolved over time.

Evolution of AI Landscape

• Google search trends on major topics like machine learning, deep learning, big data show that there has been a lot of transition in the last 20 years.

• Specialized hardware like GPUs and TPUs have improved efficiency of some models.

• Improved penetration of internet and connected devices has created a large amount of data for us to take advantage of.

• Traditional approaches relied on handcrafted rules and logical reasoning which were complex to create and maintain.

• Deep learning technology allows models to learn on their own without human intervention.

Generative AI & Large Language Models (LLMs)

The speakers discuss generative AI & LLMs.

Generative AI & LLMs

• Generative AI involves leveraging powerful foundation models to generate new data such as images, music, text or even videos.

• Large language models are massive neural networks that have been trained on ridiculous amounts of text data scoured from across the web books, articles, web pages and encyclopedias.

• GPT is one of the most advanced pre-trained models and is the foundation of chat GPT.

• LLMs can summarize text, do language translation translations, power chatbots or even generate new content.

Langchain & Production Challenges

The speakers discuss Langchain and challenges in adopting LLMs in production.

Langchain & Production Challenges

• Langchain is a framework for developing applications powered by language models.

• Other major LLM providers like OpenAI, Cohere and Hugging Face are partnering with all the different cloud players to make this technology available to developers and enterprises.

• Adopting LLMs in production is still quite a challenge due to cost. Self-hosting or training your own language model from scratch can be pretty costly.

• Using fully managed services like OpenAI or Cohere can also be expensive due to repetitive or unchecked API calls.

Quality and Security Concerns in Enterprise AI

The speaker discusses the impact of limited clarity regarding data used to train and test LLMs on model biases. This can lead to models confidently making things up when presented with unknown tasks, a phenomenon known as "hallucinating."

• Limited clarity regarding specific data used to train and test LLMs can impact model biases.

• Models may confidently make things up when presented with unknown tasks, leading to "hallucinating."

Introduction to Lane Change Library

The speaker introduces the Lane Change library, an emerging open-source project that provides building blocks for enterprise-grade applications. These building blocks include standard interfaces for model wrappers, vector store wrappers, and retrievers.

• Lane Change is an emerging open-source project that provides building blocks for enterprise-grade applications.

• Building blocks include standard interfaces for model wrappers, vector store wrappers, and retrievers.

Components of Lane Change Library

The speaker describes the different components of the Lane Change library, including both standard interface components (e.g., model wrappers) and in-house built components (e.g., question answering chains).

• Different components of the Lane Change library include both standard interface components (e.g., model wrappers) and in-house built components (e.g., question answering chains).

• In-house built components were created based on common abstractions observed among people building similar applications.

Value Propositions of Lane Change Library

The speaker discusses the value propositions of using the Lane Change library, including speed of getting started, interoperability between different pieces, and customizability.

• Speed of getting started is a key value proposition of using the Lane Change library due to its building blocks with standard interfaces and pre-built chains.

• Interoperability between different pieces (e.g., model providers, embedding providers, vector stores) makes it easy to bring your own or write your own components.

• Customizability is another value proposition enabled by the ability to swap in different components and prompts specific to your application or use case.

Introduction to Lang Chain

In this section, the speaker talks about the motivation behind creating Lang Chain and how it has evolved over time. They also discuss how the library has been adopted by developers from different backgrounds.

Motivation for Lang Chain

• The core motivation was to abstract away common patterns in machine learning.

• After a month of working on it, the speaker realized there was something exciting here.

• There was a lot of interest from web developers and people getting started with building applications.

Adoption of Lang Chain

• Language models are now easy to use through APIs, making it accessible to application developers.

• A TypeScript library was released due to interest from non-Python developers.

• The open-source nature of Lang Chain allowed for collaboration with a diverse group of people.

Redis and its Role in Large Language Models

In this section, the speaker discusses Redis and its role in large language models. They also talk about how Redis has extended its capabilities to support various data structures and developer ergonomics.

Redis as a Data Storage Solution

• Redis has been part of their ecosystem for almost 15 years and offers core data structures for storing and retrieving data.

• Redis offers supported client libraries in almost every open-source language, making it easier for developers to build applications that can take advantage of all its capabilities.

Extending Capabilities with Probabilistic Data Structures

• Redis has extended its capabilities by adding support for JSON and probabilistic data structures such as Bloom filters.

• Developer ergonomics is a priority for Redis, making it easier for developers to build applications that can take advantage of all its capabilities.

Redis Enterprise Overview

In this section, the speaker talks about how Redis is extending its capabilities by adding layers on top that introduce things like event streaming or even integration capabilities with other databases. They have also introduced query and search capabilities to allow you to inspect some of these hashes or JSON data within Redis.

Extending Capabilities of Redis

• Adding layers on top that introduce things like event streaming or even integration capabilities with other databases.

• Introduced query and search capabilities to allow you to inspect some of these hashes or JSON data within Redis.

• Introduced server-side functions so you can interact with your data within Redis in a variety of ways programmatically.

Enterprise Requirements

• Implemented linear scalability and high availability and durability so you will never lose data or even tiered memory access so that you don't have to use RAM for everything.

• Focused around the developer.

Redis as a Vector Database

In this section, the speaker talks about how Redis offers the capability directly within their database to store unstructured data as embeddings. This opens up a whole host of use cases for some really traditional pieces of technology things like recommendation systems document or information retrieval but they are really excited about some of the work that you can do with large language models.

Vector Database Properties

• Externalized knowledge base loaded with domain-specific or even unstructured data encoded as vectors or embeddings.

• Numeric representations encode meaning and context often from an AI or machine learning model on a piece of data.

• Perform semantic search for discovering similar or relevant pieces of data compared to some input.

• Vector database needs to support real-time credit operations so that as your system changes as data changes it needs to be able to make those updates to the index and to the underlying records in real-time.

Redis as a Vector Database

• Out of the box, Redis offers two vector indexing methods including flat for brute force style approach and H and SW for faster but approximate search.

• Three different search methods one is a traditional k n a n kind of approach where you can select what are the top K most relevant texts or images to my input another one is with hybrid filtering where you can use some of the existing search and query features including filters for example numeric filters or tag filters or even Tech search to combine that with Vector search and make it more powerful.

Introduction to Language Models and Redis

In this section, the speakers introduce the use of vector databases like Redis to improve large language models in production. They provide code examples for setting up a language model and Redis.

Setting Up Language Model and Redis

• The setup includes installing the Python package for the language model, which is OpenAI in this case.

• The language model can be called by passing in a string. There are two calls in the code snippets provided to demonstrate what the model is good at answering and what it might not be able to answer.

• A wrapper has been created by Tyler and CodeRedis that makes it easy to set up Redis. The wrapper abstracts away difficult parts of Redis setup.

• To create an embedding used for semantic search, we pass a list of texts, an embedding model, and two Redis variables (index name and URL).

• We now have a vector store that can be added to over time. It can be used natively within LinkChain apps.

Context Retrieval

• Language models take text as input and provide text as output. Prompt engineering is taking off as its own field.

• In this example, Wikipedia articles were chunked up into pieces, embeddings were created from them, and they were stored in Redis.

• This enables querying of the vector database for top K most relevant pieces or sources of information from these articles that can help answer questions or interact with user queries.

• An example provided on GitHub shows how context retrieval works using LangChain.

Implementing Context Retrieval in LangChain

In this section, the speakers discuss how to implement context retrieval in LangChain.

Implementation of Context Retrieval

• The implementation involves retrieving relevant text from Redis and inserting it into the prompt.

• This provides structure and guardrails on the model so that it does not make something up.

• The goal is to improve quality and ensure that nothing is hallucinated or made up.

Setting up the Retriever and Chain

In this section, the speakers discuss how to set up a Retriever and Chain using LinkChain. They explain how to pass in different quarks to use different methods and how to pass in filters for hybrid search.

Creating a Retriever and Chain

• First, create a Retriever using the vector store as a retreater.

• Different quarks can be passed in to use different methods such as Max marginal relevance.

• Filters can be passed in for hybrid search.

• Pass the Retriever into the Chain along with the LLM constructed before.

Types of Chains

• There are several types of chains that can be used depending on your needs.

• The "stuff" chain type puts all context into one prompt and sends it to the language model.

• Other chain types are available for larger or many documents.

Running the Chain

• To run the chain, call chain.run() with the query as input.

• return_source_documents=True highlights which documents were used to answer the question.

Caching with Redis Enterprise

In this section, Tyler explains how caching works with Redis Enterprise. He discusses semantic caching built on a vector database like Redis that can register hits on similar or relevant queries. This technique is useful for reducing application costs and overall system throughput and latency.

Semantic Caching

• A semantic cache built on a vector database like Redis can register hits on similar or relevant queries.

• This technique is useful for reducing application costs and overall system throughput and latency.

• It's especially useful when you have large user bases or commonly asked/repetitive questions with known answers.

• Redis Enterprise is used across Fortune 100 companies today.

Example of Caching

• Two types of caching are shown: standard caching and semantic caching.

• Standard caching does an exact match based on the string.

• Semantic caching uses a reddish semantic cache that can be created with a URL and an embedding model.

• The example shows how the semantic cache can find semantically similar queries.

Conclusion

In this section, the speakers wrap up their discussion by encouraging viewers to keep building in this space. They emphasize that it's easy to get started with tools like LinkChain and that there are many challenges but also many opportunities in this field.

Encouragement to Keep Building

• It's encouraging to keep building in this space.

• Tools like LinkChain make it easy to get started with just a few lines of code.

• There are many challenges but also many opportunities in this field.

Lang Chain and Redis Ecosystem

In this section, Harrison from Redis Labs talks about the new features coming down the pipe with Lang chain and how they are focused on going from prototyping to production. He also gives advice for getting started in this field.

New Features Coming Down the Pipe

• The field is so new that it changes week by week.

• They are focused on going from prototyping to production.

• They will be introducing a bunch of features around that.

Advice for Getting Started

• Just start building.

• It's still day zero in this kind of application building experience.

• Start building and quickly run into issues, then you're in a much better place to solve them when new stuff comes out.

Call to Action

In this section, Tyler and Harrison provide links for viewers to get involved with the Lang chain documentation and redis ecosystem. They also mention a GitHub project where they have been taking a question answering application using openai redis and Lang chain as the application development layer.

• Included two links:

• Link to the link chain documentation

• Link specifically to the ecosystem page for redis

• Also included a link to a GitHub project where they have been taking a q&a app using openai redis and Lang chain as the application development layer.