LangChain - Conversations with Memory (explanation & code walkthrough)

Memory and Lang Chain

In this video, the speaker discusses the importance of memory in chatbots and how it can be used in Lang chain. The speaker explains that people expect chatbots to have human-like qualities and respond accordingly. However, chatbots do not have a built-in memory, which makes it difficult for them to understand co-reference resolution across conversations.

Importance of Memory in Chatbots

• People treat chatbots as if they are human and expect them to have human-like qualities.

• Abbreviation is often used when referring back to things spoken about earlier on in the conversation.

• Co-reference resolution is important for large language models to pick up who or what is being referred to.

Current Options for Dealing with Memory

• Large language models do not have a built-in memory.

• Transformer models with some kind of memory hooked into them have been attempted but are not optimal at scale.

• Two main options for dealing with memory:

• Put it back into the prompt

• Have some kind of external lookup

Putting Memory Back into the Prompt

• The context is added to the prompt, allowing the language model to track what's going on in the conversation.

• However, this method has limitations as there is a limit on how much information can be stored within the span of tokens that a language model can take.

Other Methods for Dealing with Memory

• Lang Gina has several methods built-in:

• Simple method: putting memory back into prompt

• Summarizing conversation as it goes along

• Limiting window so it only remembers the last few encounters

• Merging some of those methods together

• External methods include using databases and external lookup to bring information back into the prompt.

Introduction to Memory in Lang Chain

In this section, the speaker introduces the concept of memory in Lang Chain and explains how it works.

Conversation Buffer Memory

• The conversation buffer memory is the simplest form of memory in Lang Chain.

• It tracks what the user says and what the agent responds with.

• This kind of memory is useful for situations where there are a limited number of interactions with a bot or when you want to shut down after a certain number of interactions.

Conversation Summary Memory

• The conversation summary memory summarizes the conversation rather than passing everything that is said back and forth into the prompt.

• It stores a summary of the conversation as a whole, which uses fewer tokens over time.

• This kind of memory is useful for longer conversations where you don't need to store every single interaction verbatim.

AI Customer Support

This section covers different approaches to building an AI customer support system, including co-reference resolution and conversation buffer window memory.

Co-Reference Resolution

• The AI responds positively when asked for customer support and asks what kind of support is needed.

• The AI uses co-reference resolution to refer back to the user's previous statement.

Conversation Buffer Window Memory

• The conversation is fed into the prompt, with the last number of interactions set by k.

• The prompt has the last two full interactions plus the current one.

• A short memory of three to five steps back in the conversation can be used to fool people.

• The full conversation is stored in a memory buffer.

Summary with Buffer

• A summary is created using a buffer of 40 tokens.

• The summary includes both a moving summary buffer and a Knowledge Graph (KG) memory.

• Entities are extracted from the conversation based on what was said.

Knowledge Graph Memory

• This approach extracts information from conversations in the form of entities and KG memory.

• A simple prompt is used as an example.

• Information is added to the prompt by the AI.

Extracting Information from Conversations

This section covers how to extract information from conversations using entities and KG memory.

Entity Extraction

• Entities are extracted from conversations based on what was said.

• This approach can be useful for passing information into other modules.

KG Memory Extraction

• Information is extracted from conversations in the form of entities and KG memory.

• A simple prompt is used as an example.

Introduction to Memory in Conversational AI

In this section, the speaker introduces the concept of memory in conversational AI and explains how it can be used to extract useful information from user input.

Understanding Knowledge Graphs

• Relevant information is passed into the system, which constructs a mini Knowledge Graph based on that information.

• The mini Knowledge Graph is represented as a Network X graph, with nodes representing entities and edges representing relationships between them.

• This approach can be useful for extracting information about specific topics or entities.

Using Entity Memory

• Entity memory is similar to Knowledge Graphs but focuses on specific types of entities.

• Examples of entity types can be included in prompts to help guide the conversation.

• Entity memory can extract relevant entities from user input and use them to guide the conversation.

Combining Memories

• Different types of memories can be combined to extract more complex relationships between entities.

• The speaker does not provide any further details or examples on this topic.