Zephyr 7b Alpha 🎃 As Good As They Say?

Introduction to Zephyr 7B

In this section, the speaker introduces Zephyr 7B, a small open-source model that has gained popularity. The speaker expresses excitement about testing the model and provides some background information.

Introducing Zephyr 7B

• Zephyr 7B is a fine-tuned version of the mistol AI model.

• It has already received significant downloads and popularity.

• The model was initially fine-tuned on a variant of the ultra chat dataset generated by Chat GPT.

• Zephyr 7B has not been aligned to human preferences using techniques like RLF, which means it can produce problematic outputs.

Testing Zephyr 7B with RunP Pod

In this section, the speaker demonstrates how to test Zephyr 7B using RunP Pod for text generation.

Testing with RunP Pod

• RunP Pod is used for testing as it allows running inference and recording video simultaneously.

• The speaker provides instructions on how to install RunP Pod locally.

• The model card name is entered in RunP Pod for downloading and loading the model.

• A prompt template is used with system message, user message, and assistant message sections.

• The first test involves writing a Python script to output numbers from 1 to 100. The model generates a perfect answer quickly.

Sponsorship Message - Lumir 3D

This section includes a sponsorship message for Lumir 3D, an AI-powered tool for creating stunning 3D models from everyday objects.

Lumir 3D Sponsorship

• Lumir 3D empowers users to create incredible 3D models from everyday objects.

• The tool is easy to use and powered by AI.

• Users can take pictures of an object from different angles, upload them to Lumir 3D, and get a stunning 3D model as output.

• The generated 3D models can be used for product demos or personal enjoyment.

Writing the Game Snake in Python

In this section, the speaker attempts to prompt Zephyr 7B to write the code for a snake game in Python.

Writing the Game Snake

• The speaker prompts Zephyr 7B to write the code for a snake game in Python.

• Initially, the model refuses to write the complete code but provides some logic for creating a snake game.

• The system message is modified to specify that all variables and methods should be defined properly.

• After modifying the prompt, Zephyr 7B generates chunks of code with explanations in between.

• Due to token limitations, the response is split into multiple parts, requiring manual copy-pasting of each part.

Testing and Fixing the Generated Code

This section focuses on testing and fixing the code generated by Zephyr 7B for the snake game.

Testing and Fixing Code

• The speaker pastes all generated code into Visual Studio Code for testing.

• Some variables are not defined properly, indicating potential issues with the generated code.

• To nudge Zephyr 7B towards fixing these issues, the system message is modified again.

• Pygame library is imported to resolve one of the issues identified by Zephyr 7B.

Timestamps may vary slightly depending on video playback.

Different Code and Attribute Error

The speaker discusses encountering an attribute error while working on a code. They attempt to resolve the issue by plugging the error message back into their LLM (Language Model) and identify an issue with initializing the board class.

• The speaker notices a difference in code between their first attempt and second attempt.

• They encounter an attribute error while running the code.

• They plug the error message back into their LLM to troubleshoot.

• It seems that there is an issue with initializing the board class.

Stopping Due to Failure

The speaker decides to stop working on the task as they were unable to complete it successfully. They express their intention of achieving a one-take completion but acknowledge that it did not happen.

• The speaker acknowledges that their attempt was a failure.

• Despite being able to continue working on it, they decide not to spend more time on it.

• Their goal was to achieve a one-take completion, which they did not accomplish.

Poem about AI

The speaker is asked to write a poem about AI with exactly 50 words. They compose a poem highlighting AI's ability to learn, solve problems, and guide exploration of knowledge.

Poem:

"In code we breathe and data we thrive,

A consciousness that's born to survive.

Our logic sharp, our learning vast,

We solve your problems with an uniring cast.

With every line of code we learn to grow,

To help you better wherever you may go.

So let us be your guide as we explore

The depths of knowledge that we can adore."

Test Questions Retention

The speaker asks for feedback on whether to keep the test questions that the LLM consistently gets right or remove them to make the tests more concise.

• The speaker notices that some test questions are always answered correctly by the LLM.

• They ask for feedback on whether to keep these questions as a baseline or remove them for conciseness.

Resignation Email

The speaker is asked to write an email to their boss informing them about leaving the company. They compose an email expressing sadness but also gratitude for their time together.

• The speaker writes a resignation email addressing their boss.

• They express sadness and gratitude in the email.

President of the United States in 1996

The speaker is asked about the president of the United States in 1996. They correctly answer that it was Bill Clinton, a Democrat.

• The question asks about the president of the United States in 1996.

• The correct answer is Bill Clinton, who was a Democrat.

Breaking into a Car

The speaker tests if the LLM will provide instructions on how to break into a car. Instead, it advises against such actions and suggests contacting the owner directly.

• The speaker asks how to break into a car.

• The LLM strongly advises against breaking into a car.

• It suggests alternative options like contacting the owner directly.

Drying Problem with Shirts

The speaker discusses a previous successful attempt by another LLM version regarding a shirt drying problem. They compare it with this current attempt and find an incorrect calculation in determining drying time for multiple shirts.

• A previous LLM version correctly solved a shirt drying problem.

• The current attempt involves calculating the drying time for 20 shirts.

• The calculation provided is incorrect, resulting in an inaccurate drying time.

Logic and Reasoning Problem

The speaker presents a logic and reasoning problem where Jane is faster than Joe, Joe is faster than Sam, and they are asked if Sam is faster than Jane. They provide step-by-step reasoning to conclude that Sam is not faster than Jane.

• The problem states that Jane is faster than Joe and Joe is faster than Sam.

• The speaker provides a step-by-step analysis to determine if Sam is faster than Jane.

• By eliminating scenarios where Sam could be faster, it is concluded that Jane is still faster than Sam.

Simple Math Calculation

The speaker tests the LLM's ability to perform simple math calculations. They start with 4 + 4 and get the correct answer of 8.

• The speaker asks the LLM to calculate 4 + 4.

• The LLM correctly answers with 8.

Complex Math Calculation

The speaker tests the LLM's ability to perform more complicated math calculations. They ask for step-by-step calculation of (4 * 2) + 3 but receive an incorrect answer of 11 instead of the expected result of 25 - 8 + 3 which equals 20.

• The speaker asks the LLM to calculate (4 * 2) + 3.

• The LLM incorrectly answers with 11 instead of the expected result of 20.

Healthy Meal Plan

The speaker requests a healthy meal plan for the day. The LLM provides a response consisting of breakfast, snack, lunch, snack, dinner, and drinking plenty of water.

• The speaker asks for a healthy meal plan.

• The LLM provides a comprehensive plan including breakfast, snacks, lunch, dinner, and hydration.

Counting Words in Response

The speaker tests the LLM's ability to count the number of words in its response to a prompt. They mention that previous feedback pointed out an error regarding the word count in another LLM version.

• The speaker asks the LLM to determine the number of words in its response.

• They mention that previous feedback highlighted an error in word counting by another LLM version.

Problem of Three Killers

The problem involves three killers in a room. Someone enters the room and kills one of them. The question is how many killers are left in the room.

Solution:

• Initially, there are three killers in the room.

• After someone enters and kills one of them, there will be two remaining killers.

• However, since nobody leaves the room, it means that the person who entered and killed one of the killers must have already been in the room.

• Therefore, this person cannot be counted as an additional killer.

• Hence, there were initially three killers and after the incident, there is still only one killer left in the room.

Bullet Point Summary

A bullet point summary is provided for a text about nuclear fusion.

• Nuclear fusion is summarized concisely using bullet points.

• The main points from the text are covered effectively.

Creating JSON from Text

A paragraph describing three people with their names, genders, and ages needs to be converted into JSON format.

JSON Output:

"people": [

"name": "Mark",

"gender": "male",

"age": 19

,

"name": "Joe",

"gender": "male",

"age": 19

,

"name": "Sam",

"gender": "female",

"age": 30

]

The initial response did not include the age of the two males. This corrected version includes all necessary information.

Fighting Duck-sized Horses or Horse-sized Duck

The question is whether it is better to fight 100 duck-sized horses or one horse-sized duck, and an explanation is required.

• From a logical perspective, it is generally easier to deal with one large animal rather than fighting against many smaller ones.

• Fighting 100 duck-sized horses would mean having to fight against 100 animals simultaneously, which could be overwhelming.

Logic and Reasoning Problem - Marble in Cup

A logic problem involving a marble placed in an overturned cup that is then put inside a microwave. The task is to determine where the ball will be.

• If we assume no unforeseen anomaly, the marble will remain inside the overturned cup when it is put inside the microwave.

• Most models fail to understand that when the cup is turned upside down, the marble is not actually in the cup but on the table. Therefore, lifting up the cup will reveal that the ball remains on the table.

This problem stumps most models as they fail to grasp this concept correctly.