MemGPT + Open-Source Models Tutorial 🔥 Insane Power

Introduction and Setting Up the Model

In this section, the speaker introduces the topic of using an open-source local model with MGPT. They discuss the recent updates made by the team behind MGPT and explain how to set up the model on Runpod or a local machine.

Setting Up on Runpod or Local Machine

• To run the model on Runpod, click on "Secure Cloud" and select a GPU.

• Customize deployment if needed, then click "Continue" and "Deploy".

• Wait for the model to load fully, then click "Connect" and "Connect to HTTP service".

• Download the Dolphin 2.0 mistol 7B model from the Model tab in Text Generation Web UI.

• Alternatively, follow instructions for installing Text Generation Web UI on your local machine.

• Use Charles and Vivien's recommended aeroboros prompt template for best results.

Loading and Configuring the Model

This section covers loading and configuring the downloaded model using MGPT.

Loading the Model

• Use Model Loader Transformers to load the unquantized model.

• Click refresh to load up the downloaded model.

• Select none first before switching back to choose a new model.

• Click "Load" to successfully load the model into memory.

Configuring Backend Type

• Switch to Session tab in Text Generation Web UI.

• Remove OpenAI flag enabled previously when using autogen with a local model.

• Set backend type as web UI for default API provided by Text Generation Web UI.

Installing MGPT and Running Locally

This section explains how to install MGPT module, configure API endpoint URL, create a conda environment, install requirements, and run the local model.

Installing MGPT

• Clone the MGPT repository from GitHub or use the provided MGPT module.

• Change directory to mgpt using cd mgpt.

• Export the URL of the Runpod instance as export openaiore_api_base="URL:5000".

• Install requirements using pip install -r requirements.txt.

Running Locally

• Run python3 main.py --noopenai_verify to load and run the local model.

• Ensure that a config file is found for successful execution.

Conclusion

The speaker concludes by mentioning that this technology is cutting-edge and may have bugs. They express gratitude to Charles and Vivien for their assistance.

New Section Running the API Endpoint

In this section, the speaker demonstrates how to run the API endpoint without preloading anything and shows it working.

Running the API Endpoint

• The speaker mentions that they will not go into a specific use case but will show the API endpoint working.

• By hitting enter, the new runpod opsource API endpoint is triggered.

• The demonstration successfully hits the API endpoint.

New Section Deep Dive on Mem GPT

The speaker discusses their plan to create a deep dive video on getting the most out of Mem GPT and asks for feedback from viewers on whether they should prioritize this video.

Deep Dive on Mem GPT

• The speaker mentions their intention to create a deep dive video on maximizing the usage of Mem GPT.

• They ask viewers to provide feedback in the comments section regarding their interest in prioritizing this video.

• The speaker expresses uncertainty about which videos to record first and seeks input from viewers.

New Section Multi-Line Support and Saving User's Name

This section highlights multi-line support in Mem GPT and demonstrates how to save a user's name using this feature.

Multi-Line Support and Saving User's Name

• The speaker explains that although hitting enter creates a new line, it does not complete the action. Instead, pressing Escape followed by enter is required.

• This process triggers Mem GPT to save the user's name as "Matt."

• To verify if the name was saved correctly, asking "What's My Name" returns "Matt."

New Section Conclusion and Upcoming Video Topic

The speaker concludes by mentioning an upcoming video topic related to setting up autogen with Mem GPT.

Conclusion and Upcoming Video Topic

• The speaker expresses excitement about recording a video on setting up autogen with Mem GPT.

• They encourage viewers to like the current video and anticipate the upcoming one.