AI Agent Dev School 1 pt 1

Live Stream Introduction and Overview

Streaming Platforms and Initial Setup

• The stream is live on multiple platforms including Twitch, YouTube, and X. Confirmation of the stream's functionality is requested.

• The host introduces themselves as "Moon makes Magic" on YouTube and mentions plans to cover topics like npm, Node.js, TypeScript, and creating an agent in TypeScript.

Audience Engagement and Discord Interaction

• Viewers are encouraged to join the Discord space for real-time interaction; the host will respond to questions from Discord users during the stream.

• The host emphasizes that they cannot see comments from other platforms but will focus on interactions within Discord.

Development Basics: Tools and Concepts

Introduction to Development Tools

• The first part of the session will cover development basics such as Node.js, TypeScript, npm installation, WSL for Windows users, pnpm installation, cloning repositories, switching branches, and basic git commands.

• A character creation demonstration is planned along with running that character in a Discord environment while ensuring security by not leaking keys.

Q&A Session Preparation

• After covering basic setup and character creation, there will be a Q&A session focused on these foundational concepts.

Agent Concepts: Understanding Architecture

Documentation Review

• The session will include reviewing documentation sites related to agent concepts and discussing code architecture.

• Key abstractions that make Eliza unique will be highlighted alongside standard practices in agent development.

Evaluators vs. Reflection

• Discussion includes differentiating between evaluators and reflection within agents; insights into how these concepts relate to each other will be provided.

Practical Application: Building Actions

Action Creation Examples

• Plans include creating actions such as fetching news via an API or extracting personality scores based on user input.

• A simple approach using JSON files instead of databases may be demonstrated for clarity in understanding complex abstractions.

Final Preparations Before Start

Stream Logistics

• The host reminds viewers about the time constraints of three hours for content delivery due to a hard stop at 9 o'clock.

Recording Information

Introduction to Agent Dev School

Overview of the Session

• The session begins with an invitation for participants to share links on Discord, indicating a collaborative environment.

• The host acknowledges that the first hour may be basic for experienced developers but emphasizes its importance for beginners.

• A three-hour stream is planned, aiming to provide foundational knowledge and encourage self-learning among participants.

Goals and Structure

• The format will include tutorials followed by practical work sessions, such as creating plugins or integrations with partners.

• Emphasis is placed on sharing knowledge and skills that can lead to significant personal and financial growth through AI development.

The Journey of JavaScript

Historical Context

• Shaw introduces himself as an experienced developer who has been coding since childhood, highlighting his passion for JavaScript and TypeScript.

• Discussion shifts to Brendan Eich, the creator of JavaScript, noting his rapid development of the language in just ten days.

Evolution of JavaScript

• Initially designed for basic interactivity in browsers, JavaScript's capabilities have expanded significantly over time.

• The transition from client-side scripting to server-side applications began when developers realized they could run JavaScript outside the browser.

Node.js: Revolutionizing Development

Introduction to Node.js

• Node.js emerged as a runtime environment allowing developers to use JavaScript on servers, leveraging the V8 engine from Chrome.

• Ryan Dahl initiated Node.js as an open-source project in 2009, which has since become integral to modern web development frameworks like React and Express.

Community Engagement

Understanding Node.js and TypeScript

Introduction to Node.js

• Node.js is a runtime environment that allows you to run JavaScript code outside of a web browser, enabling the development of server-side applications.

• It can compile applications into executable formats (like .exe), but typically developers clone and run existing Node.js applications directly.

The Flexibility and Challenges of JavaScript

• JavaScript is known for its flexibility, allowing developers to perform complex tasks quickly; however, it lacks computational efficiency.

• One major drawback is the absence of types in JavaScript, which can lead to issues when defining data structures or objects.

Introduction to TypeScript

• TypeScript addresses the lack of typing in JavaScript by introducing type definitions, enhancing code reliability and maintainability.

• Types can be defined for various entities (e.g., fruit with properties like name and color) and can also have subtypes (e.g., Apple with an additional property isSweet).

Importance of Typing

• By defining types, developers ensure that variables adhere to specific structures, reducing errors during development.

• This concept stems from type theory in mathematics, where types act as contracts that enforce certain rules on data structures.

Comparison with Statically Typed Languages

• Statically typed languages like Rust or C++ check types at compile time, providing immediate feedback if there are discrepancies.

• In contrast, JavaScript's dynamic typing allows for more freedom but increases the risk of runtime errors due to undefined properties.

Common Issues in JavaScript Development

• Developers often face challenges when accessing properties on objects without guaranteed structure; this leads to potential undefined values.

Understanding TypeScript and Its Advantages

The Nature of JavaScript and Python

• JavaScript is untyped, which can lead to flexibility issues; similar problems exist in Python, known for its magical flexibility but also being fundamentally untyped.

• Python has introduced native types through typings packages, a trend that is slowly emerging in JavaScript as well.

Introduction to TypeScript

• Microsoft designed TypeScript to provide a type system for JavaScript, allowing developers to guarantee certain properties about their code.

• TypeScript includes a type checker that helps prevent compilation errors by ensuring the validity of types before building the code.

Benefits of Using TypeScript

• TypeScript offers static type safety akin to languages like C++, C, and Rust while compiling down into standard JavaScript.

• It has gained immense popularity among developers using frameworks like Next.js or React due to its typing capabilities.

Applications Built with TypeScript

• Many widely-used applications (e.g., Discord, Microsoft Teams, Zoom) are built using TypeScript within a JavaScript runtime environment.

• These applications often connect with native runtimes (like React Native), utilizing plugins for more complex functionalities written in languages such as C++ or Swift.

Emerging Technologies and Trends

• Electron is highlighted as a popular framework for building apps on Chromium; newer technologies like Tauri (Rust-based) are gaining traction for cross-platform app development while still leveraging TypeScript on the front end.

• Tauri is recommended as an innovative way to build applications that run anywhere while maintaining compatibility with modern web technologies.

Flexibility and Performance of TypeScript

• The speaker emphasizes that despite many agent developers using Python, TypeScript excels at handling streaming data and HTTP requests effectively within browser environments.

Development Tools and Techniques

Overview of Development Preferences

• The speaker discusses their preference for using asynchronous programming over synchronous typing, noting a better overall experience in development.

• They highly recommend using Cursor, a tool that automatically completes code, enhancing productivity by allowing users to request transformations (e.g., turning code into poetry).

Favorite AI Tools

• The speaker expresses a strong preference for Claude, an AI model they find more usable compared to OpenAI's models. They mention the ability to switch between different models within Cursor.

• They highlight the advantages of Claude's long context capabilities, which allow users to input multiple files and perform complex tasks.

Code Management and Tooling

• Introduction of Code to Prompt, a tool that provides a source tree of files and source code, facilitating easier coding processes.

• The speaker emphasizes the utility of Claude as a "Junior engineer," recommending it for interpreting and explaining code.

Installation and Package Management

• Discussion on the importance of package managers in JavaScript development, emphasizing that most applications are built upon existing code libraries.

• The speaker notes that 99% of app functionality is derived from pre-written code available through packages, illustrating this with examples from their own project dependencies.

Utilizing Node.js and npm

• An explanation of Node.js as a runtime environment for executing JavaScript applications, along with its package management system (npm).

Understanding the Node Package Manager and Monorepos

Introduction to npm and Package Ecosystem

• The speaker discusses integrating a package called L chain into their project, highlighting the vast ecosystem of JavaScript packages available through npm, which currently hosts over 3.3 million packages.

• The speaker explores the TypeScript bindings for L chain, emphasizing the wealth of information and resources available for developers within these packages.

The Power of Open Source Contributions

• Acknowledges the collaborative nature of open-source development, where numerous developers contribute to creating powerful tools that can be easily integrated into projects.

• Mentions Python's package index (PyPI), drawing parallels between JavaScript and Python ecosystems in terms of accessibility and community contributions.

Utilizing npm for Project Management

• Highlights how simple commands like pnpm install L chain allow developers to effortlessly incorporate complex functionalities into their applications without needing deep knowledge of each package.

• Introduces npm as a tool not only for downloading packages but also for managing project scripts and dependencies effectively.

Challenges with Monorepos

• Discusses the concept of monorepos, where multiple packages are housed within a single repository, making it easier to manage interdependencies among them.

• Explains that while separate repositories could be created for each package, using a monorepo simplifies collaboration across different teams working on various components.

Issues with npm in Monorepo Management

• Points out a significant drawback: npm installs duplicate copies of packages in every node_modules folder across all projects, leading to bloated file sizes and longer installation times.

Understanding pnpm and WSL for Node.js Development

Benefits of Using pnpm

• pnpm is designed to improve the development experience by sharing dependencies across projects, eliminating the need to reinstall them for each project.

• It enhances performance compared to npm, which can struggle with package linking in monorepo setups.

Windows Subsystem for Linux (WSL)

• Developing with Node.js and Python is generally easier on Linux than on Windows, especially due to dependency management issues.

• Microsoft created WSL to address developers' frustrations with building applications on Windows; WSL2 is recommended for better functionality.

• WSL allows users to run a full Linux environment on their Windows machine, making development smoother and more intuitive.

Advantages of Using WSL

• The command line experience in Windows differs significantly from that in Linux or macOS, often leading to confusion; using WSL mitigates these issues.

• Installing WSL takes about 30 minutes and greatly improves compatibility when working with various programming languages like Python or JavaScript.

Installing Node.js via NVM

• To install Node.js effectively, it's recommended to use NVM (Node Version Manager), which simplifies managing multiple versions of Node.js.

• NVM can be installed using a curl command that downloads an installation script directly from GitHub.

Managing Node Versions

• Once NVM is installed, it allows users to easily switch between different versions of Node.js as needed for various projects.

Getting Started with Eliza and pnpm

Introduction to npm and pnpm

• Node.js now includes npm by default, allowing easy package installation without additional setup.

• pnpm must be installed separately; it is not included with Node.js.

• For Windows users, PowerShell can be used to install pnpm via the Invoke-WebRequest command.

Setting Up the Eliza Starter Kit

• The Eliza starter kit provides a pre-configured environment for building agents without modifying existing code.

• Users need to copy the .env.example file to .env, install dependencies using pnpm, and start the application with pnpm start.

Development Options

• There are two main approaches: using the starter kit for quick setups or cloning the core repository for deeper development.

• The starter repo is lightweight, allowing users to focus on creating their own agents without altering core functionalities.

Package Management in Eliza

• The package.json file lists all imported packages from npm, which are automatically managed during installation.

• Users can create plugins or agents independently of the core codebase, promoting modular development.

Collaboration and Code Contributions

• New pull requests allow contributors to submit changes easily; these can include custom character files or enhancements.

• Merging contributions involves reviewing changes for safety before integrating them into the main project.

Conclusion of Setup Instructions

Getting Started with the Eliza Repo

Setting Up the Environment

• The speaker discusses editing the environment variable file and starting scripts using pnpm run start, emphasizing that this is part of getting started with the project.

• After cloning a repository, it's essential to install all packages listed in package.json using pnpm install, which contains all necessary code imports for the project.

Cloning and Using Starter Kits

• The speaker plans to clone the Eliza repo from scratch and mentions that starter kits can simplify tasks like tweeting but are less customizable.

• Basic Git operations will be covered, including switching branches and pulling changes, to help users get familiar with version control.

Introduction to Git and GitHub

• A brief history of Git is provided, mentioning its creator Linus Torvalds and how it was developed as a solution for managing contributions from multiple developers on Linux.

• The necessity of a simple version control tool led to the creation of Git, allowing collaboration among developers without complex setups.

Understanding GitHub's Role

• GitHub emerged as a platform that hosts Git repositories, making it easier for users to manage their projects without needing their own servers.

• The popularity of GitHub grew as it allowed users to manage operations efficiently while providing features like issue tracking and pull requests (PR).

Features of GitHub

• Users can report bugs or suggest features through issues on GitHub. They can also search existing issues before submitting new ones.

• Pull requests allow contributors to submit code changes for review. This process includes discussions about proposed changes within the community.

Project Management and GitHub Features

Current Iteration Focus

• Emphasis on focusing on the current week's tasks and deliverables, highlighting a structured approach to project management.

• Discussion of creating a roadmap to visualize weekly goals and issues, enhancing clarity in project tracking.

Setting Up Actions in GitHub

• Introduction to setting up automated actions in GitHub, such as deploying the latest version upon committing changes.

• Importance of understanding basic commands like git, GitHub, and npm for effective navigation within the ecosystem.

Documentation and Resources

• Mention of GitHub Pages hosting documentation sites, providing users with comprehensive resources for project guidance.

• Encouragement to utilize AI tools for navigating documentation efficiently, making it easier for users to find specific information.

Understanding Forking Repositories

• Explanation of how to fork repositories on GitHub, allowing users to create their own versions while maintaining a connection to the original repository.

• Description of syncing forks with upstream changes, enabling independent development while staying updated with the main project.

Community Collaboration through Pull Requests

• Insight into viewing insights from forks and contributions made by various developers within the community.

Cloning a Repository and Setting Up the Environment

Cloning the Repository

• The speaker discusses options for cloning a repository, mentioning that users can either fork it or clone directly without worrying about edits.

• The speaker explains how to use GitHub CLI for cloning, noting that if Git is installed on Linux machines, one can simply use git clone with the repository URL.

• Alternatives like SSH are mentioned, along with GitHub Desktop as a user-friendly option for those uncomfortable with command-line interfaces.

Downloading vs. Cloning

• The speaker advises against downloading a zip file of the repository due to lack of source control; editing downloaded files could lead to irreversible changes.

• Emphasizes using GitHub Desktop or cloning instead of downloading zip files to maintain connection with source control.

Installing Dependencies

• After cloning, users need to navigate into the cloned directory using cd, followed by opening it in an editor (e.g., Visual Studio Code).

• The installation of packages is initiated using pnpm install, which installs all dependencies listed in package.json.

Managing Branches and Releases

• The speaker highlights potential issues when working directly from the main branch due to rapid changes; stable releases are recommended instead.

• Users are encouraged to check release notes for stable versions rather than relying on potentially unstable main branches.

Cleaning and Reinstalling Packages

• To ensure compatibility after switching branches, running pnpm clean is suggested before reinstalling dependencies.

• This cleaning process helps remove any outdated or conflicting installations before executing another pnpm install.

Running the Application

Understanding Node.js Project Setup

Importance of Proper Setup in Node.js Projects

• The speaker emphasizes the necessity of ensuring proper setup when cloning repositories, using a Next.js starter as an example. They highlight that Next.js is a full-stack application framework utilizing JavaScript, TypeScript, and React.

• The discussion points out that many developers overlook essential scripts in the package.json file, such as "dev" and "build," leading to common pitfalls when running projects without understanding their structure.

• Build errors often arise from not executing the build process first. The speaker notes that these errors are typically due to missing types and stresses the importance of following installation instructions carefully.

• The need for building all packages before running the project is reiterated. The speaker mentions potential improvements to speed up this process but acknowledges current limitations.

• After building, users can run either "start" or "dev." The latter allows for a watch mode that automatically updates dependencies, although this feature is still under development.

Troubleshooting Common Issues

• An error occurs due to a missing API key for OpenAI's bearer authentication. This highlights the importance of having necessary credentials configured correctly before running applications.

• When starting the application with npm run start, it registers services and actions successfully. However, functionality depends on having valid API keys; otherwise, it defaults to local models.

Key Components of Any Node.js Project

• The speaker discusses foundational elements required for any Node.js project: Git, GitHub, npm (or pnpm), and Node Version Manager (NVM). These tools are crucial for managing versions and dependencies effectively.

• Acknowledgment is given that while these components are standard across various frameworks, unique aspects exist within specific projects which will be explored further in subsequent discussions.

Unique Concepts in Agent Framework Development

• Transitioning into more specialized topics related to agent frameworks, the speaker plans to discuss core concepts shared across different frameworks while also addressing their unique interpretations within their own project context.

Learning Through Experience

• A critical insight shared by the speaker emphasizes investing time in learning rather than struggling through tasks without knowledge. They advocate for dedicating time to learn skills thoroughly instead of attempting tasks blindly.

How to Learn Programming Effectively

The Importance of Self-Learning

• Emphasizes the superpower of self-learning, particularly through platforms like YouTube, where one can learn anything, including programming.

• Highlights a specific GitHub tutorial on using Git for version control tasks such as cloning repositories and submitting pull requests.

Utilizing Tools for Learning

• Discusses practical usage of Git commands and encourages learners to seek out information when unsure about commands.

• Mentions the utility of AI tools like OpenAI's ChatGPT for asking questions while also stressing the importance of watching other developers build projects on YouTube.

Recommended Learning Resources

• Suggests searching for tutorials on popular technologies like TypeScript and Node.js, indicating that these are commonly used together in development.

• Shares personal experiences with learning programming through online resources rather than traditional schooling, criticizing outdated curriculums focused on languages like Java.

Expanding Knowledge Beyond Basics

• Encourages exploring various programming languages and frameworks (e.g., Rust, Solana contracts), asserting that anyone can become proficient with dedication.

• Reinforces the idea that learning is akin to acquiring skills in a video game; persistence leads to mastery.

Understanding Low-Level Programming Concepts

• Addresses the relevance of lower-level languages (C, Assembly) in machine learning and performance-critical applications.

• Explains how high-level languages often run on low-level code (like C++) for efficiency in machine learning tasks.

Practical Applications and Insights

• Discusses assembly language's role in performance optimization within software development, citing historical examples from gaming.

• Advocates for understanding assembly language as it provides insights into what happens at a lower level during program execution.

Conclusion: Continuous Learning Mindset

Steps for Cloning and Setting Up Eliza Test

Cloning the Repository

• To clone the Eliza test repository, use the command: git clone https://github.com/a16z/Eliza-test. This will create a local copy of the repository.

Installing Dependencies

• After cloning, navigate into the directory using cd ela-2 and run pmpm install followed by pmpm run build to set up the project. Documentation is available for step-by-step guidance.

Creating Character Files in Eliza

Overview of Character Files

• The character files used in this framework can be imported into any compatible Asian framework. They are not exclusive to this project.

Default Characters

• The default characters include "Tate" and "Trump," with "Eliza" being part of the core package.

Understanding Character Types

Type Definitions

• In TypeScript, each character has a defined type that outlines its properties. Optional properties are indicated with a question mark (e.g., an ID may or may not be present).

Required Properties

• A character must have a name; other properties like username and system prompt are optional. If absent, defaults will apply.

Model Providers for Characters

Available Model Providers

• Various model providers can be utilized including OpenAI, Anthropic, Grok, Llama Cloud, Google Gemini, Vertex AI, and others. Each provider offers different capabilities for generating completions.

Decentralized Options

• Some options allow running models locally or through decentralized networks (e.g., Hus), providing free inference services.

Future Integrations and Recommendations

Expanding Model Provider Options

• Plans to integrate more model providers like Infura and Bit Tensor are underway to enhance functionality.

Recommendations on Model Usage

• The speaker expresses dissatisfaction with OpenAI's models for character generation but recommends alternatives such as Claude from Anthropic or Llama Cloud due to their superior performance.

Contextual Capabilities of Models

Token Context Window Comparison

• Gemini boasts a 2 million token context window which allows extensive input compared to Claude's 200k tokens and OpenAI's limitations. This feature enables comprehensive document analysis.

Managing Secrets in Character Settings

Environment Variables Management

• While environment variables can manage API keys across characters, upcoming features will allow secrets to be namespaced by character names for better organization.

Understanding System Prompts and Bio Randomization in AI Agents

Overview of Secrets and System Prompts

• The discussion begins with the concept of storing secrets, such as API keys, within a designated area for security purposes.

• The speaker explains that system prompts are akin to custom instructions found in OpenAI settings, which can be edited but are often left unchanged by users.

• There is mention of personal preferences influencing the system prompt, although the exact location of these settings is unclear.

Bio Randomization Techniques

• The speaker emphasizes the importance of randomness in bios to enhance uniqueness and prevent repetitive responses from AI agents.

• A lower temperature setting is discussed as a means to reduce hallucinations while maintaining creativity without increasing model parameters excessively.

• The process involves preparing context by randomizing multiple bios before generating responses, ensuring variability each time an agent interacts.

Entropy and Creativity in Responses

• By combining different bios into a single response context, the aim is to create diverse outputs that reflect various aspects of personality or character traits.

• Entropy is defined as a measure of randomness; it helps avoid settling into predictable patterns during interactions with AI models.

• The goal is to draw from different parts of the model rather than relying on a singular narrative thread, enhancing creative output.

Lore Integration and Retrieval-Augmented Generation (RAG)

• Lore functions similarly to bios; it adds depth by pulling relevant background information for character development during conversations.

• Currently, lore retrieval may not be fully optimized but aims to provide interesting context about characters based on user queries.

• RAG (retrieval augmented generation), described as searching for information before generating responses, enhances relevance and accuracy in AI outputs.

How to Create Knowledge Items Using RAG

Understanding RAG and Vector Embeddings

• The process involves pulling specific data (e.g., "Eliza") and using it to generate messages, leveraging Retrieval-Augmented Generation (RAG) techniques.

• Various algorithms exist for searching data efficiently, with some requiring less computational power while remaining valid for database retrieval.

Creating Knowledge Items

• An example is given where knowledge items are created manually from written pieces rather than videos, emphasizing the importance of text-based information.

• The speaker discusses creating multiple knowledge pieces, such as one on the "candlemaker petition," indicating a selective approach to content curation.

Tools for Knowledge Extraction

• Mention of tools like "folder to knowledge" that automate the extraction of information from PDFs into structured knowledge documents.

• Users can download their Twitter archives to create character files, showcasing how personal data can be transformed into usable formats.

Enhancements in Knowledge Ingestion

• Future improvements will allow automatic ingestion of knowledge from folders or platforms like Discord, enhancing user interaction with AI agents.

• The system will enable users to add documents directly into the agent's memory store, making searches more relevant and efficient.

Agent Communication Examples

• Discussion on creating message examples that reflect how agents communicate; these include user interactions and potential actions taken by the agent.

• The use of macros allows for dynamic conversation simulations between users and agents, providing insights into conversational flow.

Structuring Agent Responses

• Conversations are compacted down into examples that guide how an agent should respond based on previous interactions.

• Emphasis on varying response styles (witty vs. soft), which can be reflected in training data used for agent communication models.

Post Examples vs. Message Examples

• Distinction made between message examples used in interactive chats versus post examples intended for social media platforms like Twitter.

Twitter Scraper and Post Generation Tool

Overview of the Twitter Scraper

• The creator of Beth AI, named Lucid, developed a Twitter scraper tool that allows users to input their credentials (username and password) without needing an API.

• Users can scrape tweets from specific accounts (e.g., "P Marka") until they hit rate limits, making it a practical solution for gathering large amounts of tweet data.

Utilizing Scraped Data

• The scraped tweets are compiled into post examples that serve as references for generating new tweets.

• Adjectives and topics can be incorporated into different post templates to enhance creativity in content generation.

Importance of Style in Content Generation

• Style is crucial in shaping the context of generated responses; it influences how the AI interacts based on previous conversations.

• Instructions for style should be clear and concise, with less than 10 directions randomized to maintain variety while ensuring consistency.

Enhancing Creativity through Directions

• For more creative outputs, having numerous style directions is beneficial. This allows the model to produce varied lengths and types of responses based on current context.

• Chat styles are included in interactions across platforms like Discord and Telegram but not in direct Twitter posts.

Character Creation Process

How to Pass Characters in a Command Line Tool?

Passing Character Arguments

• The speaker demonstrates how to pass character arguments using the command pnpm run start --characters= followed by specific character names, such as "tate" and "trump," noting their humorous nature.

• An error occurs indicating "no such file or directory," which is attributed to a path issue. The speaker suggests correcting the path with ../ to navigate correctly.

• Clarification on the package structure: the characters are relative to the agent package, emphasizing that understanding this structure is crucial for troubleshooting errors.

• The speaker reiterates the need for correct pathing when passing in characters, highlighting that an incorrect reference leads to errors about missing files.

• After fixing the path issue, there’s anticipation of running successfully but acknowledges potential delays in processing input into a vector format.

Vector Processing and API Keys

• The process involves converting input text (e.g., "hey what's up") into a numerical vector for further operations like searching knowledge bases.

• Mention of needing an embedding model and an API key for services like Claud; highlights that without it, certain functionalities will not work.