NEW ORCA-Mini 🐳 Open-Sourced LLM that You can RUN Locally

Introduction and Overview

The speaker expresses surprise at the uncensored nature of the content and praises the model's ability to follow instructions. They mention a recent paper called "Orca Progressive learning from complex explanation traces of gpt4" that generated excitement. The paper introduced a student-teacher learning approach and a new data set creation method, demonstrating that the model can outperform ChatGPT and approach GPT-4 on certain tasks. However, neither the model nor the data set was released.

Orca Mini Model

A new model called Orca Mini has been created using the data set creation approach presented in the Orca research paper. The authors used 15 out of 16 system messages from the Orca paper to curate a unique data set. Three different OpenLlama models were trained using modified data sets, ranging from 3 billion to 13 billion parameters. This section focuses on the 7 billion parameter model.

Data Set and Training Process

The data set consists of three distinct sets: Visit LM dataset (70,000 examples), Pack I dataset (52,000 examples), and Dolly V2 dataset (15,000 examples). These data sets were modified using the approach described in the Orca research paper, with emphasis on incorporating system messages. Training was performed using eight A100 GPUs at a total cost of around $84.

Running Orca Mini Model in Jupyter Notebook

To run these models in a Jupyter notebook, import inlama for causal LM and lamba tokenizer. Use the provided helper function generate_text which takes a system message, instruction, and user input to generate a response. The model ID for the 7 billion parameter Orca Mini model is "rcomp/mini-7B". Set the tokenizer with the same model ID. Encode the prompt using the tokenizer and set parameters such as temperature. Run the model with the input parameter and prompt to get an output.

Example Prompt in Jupyter Notebook

An example prompt is provided where a letter is written to OpenAI CEO Sam Altman requesting the release of GPT-4 as an open-source model. The generated response emphasizes OpenAI's mission, encourages collaboration and innovation, and promotes transparency and accountability in AI development.

Installing and Running Orca Mini Model in Uber GPT Text Generation Web UI

To install and run the Orca Mini model in Uber GPT Text Generation Web UI, paste the model ID and click download. The downloaded model can be used within the web UI for text generation.

Timestamps are approximate and may not align perfectly with specific sections of the transcript due to differences in transcription length.

Understanding the System Prompt and User Message

In this section, the speaker discusses the system prompt and user message used in the examples. The system prompt is about being an AI assistant that helps people find information, while the user message asks for an explanation of homophones in sentences.

System Prompt and User Message

• The system prompt is "You are an AI assistant that helps people find information."

• The user message is "Explain the correct usage of homophones in the following sentences: 'The principle is your call' and 'The site at the site was quite a sight.'"

The model's response provides a definition of homophones and correctly identifies "principal" as a homophone of "Paul" in the first sentence. It also correctly identifies "site" and "sight" as homophones in the second sentence.

Overall, the model did a pretty good job with this prompt.

Generating Ideas for an Original Superhero

This section focuses on generating ideas for an original superhero, including their backstory, powers, weaknesses, and arch-nemesis.

Superhero Idea

• The generated superhero is named Phoenix.

• Phoenix has a backstory involving scientist parents who stole a powerful invention related to time manipulation.

• Her powers include controlling time on a molecular level by slowing it down or speeding it up.

• Phoenix's biggest weakness is losing control of her power, which can cause catastrophic events.

• Her arch-nemesis is called The Time Master.

This superhero idea shows creativity and interesting powers.

Testing Programming Tasks

This section explores how well the model performs with simple programming tasks.

Uploading File to S3 Bucket (Python)

• The provided code is functional but may require some spacing adjustments due to limitations in the underlying model.

The model generates HTML code for a simple web page with a single button. However, the background color change functionality is not working as expected.

An updated prompt asks for both background color change and displaying a random joke on the website. The generated code only includes the joke functionality, not the background color change.

While the model's size is impressive, it's important to choose the right system prompt for specific applications and be explicit in instructions.

Conclusion

The transcript covers various topics related to using an AI language model. It discusses understanding system prompts and user messages, generating ideas for superheroes, and testing programming tasks. The model shows promise but also highlights the importance of selecting appropriate prompts and providing clear instructions.

Model Overview

In this section, the speaker presents an overview of the Orca paper and encourages viewers to like the video and subscribe to the channel. The speaker also mentions an active Discord community for those interested in generative AI.

Introduction

• The speaker introduces the Orca paper.

• Encourages viewers to like the video and subscribe to the channel.

• Mentions an active Discord community for generative AI enthusiasts.

Please note that this is a brief summary of the transcript provided.