Prompt Engineering in the Real World with PromptLayer

Introduction

In this section, Jared Zonnerich introduces himself and the topic of prompt engineering.

• Jared Zonnerich from Prompt Layer will be giving a talk on prompt Engineering in the real world.

• Prompt engineering is a new thing that is important to know about.

• The talk will cover what prompt engineering is, how it works, and how it's going to change in the future.

Company Logo and Social Media Handles

In this section, Jared Zonnerich shares his company logo and social media handles.

• The company logo is a cake emoji.

• They had a cake at their last happy hour.

• You can follow Jared on Twitter at @I'mJaredZ.

• The website for Prompt Layer is www.promptlayer.com.

Talk Sections Overview

In this section, Jared Zonnerich provides an overview of the different sections of his talk.

• The talk will be divided into four different sections.

• The first section covers what prompt engineering is and how it works.

• The second section covers how prompt engineering is done today.

• The third section covers the challenges associated with prompt engineering today.

• The fourth section includes a demo of Prompt Layer's solution to some of these challenges.

Definition of Prompt Engineering

In this section, Jared Zonnerich defines what prompt engineering is and its importance.

• Prompt engineering involves creating prompts to get desired outputs from large language models (LLMs).

• LLMs are AI models that complete text based on input prompts given by users.

• Tinkering with prompts helps improve communication between humans and AI.

• Prompt engineering is a new field that requires tinkering and experimentation.

Large Language Models

In this section, Jared Zonnerich explains what large language models are and how they work.

• Large language models (LLMs) are AI models that complete text based on input prompts given by users.

• GPT is an example of an LLM used for chatbots.

• Tinkering with prompts helps improve communication between humans and AI.

Importance of Prompt Engineering

In this section, Jared Zonnerich emphasizes the importance of prompt engineering despite advancements in LLMs.

• Even though LLMs are getting better, prompt engineering is still important because it helps improve communication between humans and AI.

• Humans also have communication problems from time to time, so prompt engineering can help bridge the gap.

Getting Started with Prompt Engineering

In this section, Jared Zonnerich provides tips on how to get started with prompt engineering.

• The best way to get started with prompt engineering is to go to Playground and start playing around.

• Prompt engineering is a new field that requires tinkering and experimentation.

Prompt Engineering and Concepts

The speaker discusses how prompt engineering is done today and introduces the concept of chains, agents, and plugins.

Prompt Engineering Today

• Engineers test a few prompts against 10 examples on their computer before shipping it to production.

• This method is unreliable and not as thorough as traditional deployment processes.

• Companies building LMs are doing the same thing but this will hopefully change in the future.

Concepts for Prompt Engineering

Chains

• Chains are built on the concept of chaining prompts together to solve more complicated problems.

• Chaining multiple prompts together can be necessary for solving complex problems that require summarization or chunking up information.

Agents and Plugins

• Agents and plugins are used to bring in different tooling like Wolfram to help with tasks that LMs are not good at.

• LMs have solved natural language but cannot be trusted with information or computation.

Understanding the Challenges of LLMs

In this section, the speaker discusses the challenges associated with using LLMs in real-world applications. The three main challenges discussed are detecting breakages, comparing different prompts, and optimizing LLM models.

Breakages

• Detecting when things go wrong is a non-trivial problem in LLMs.

• Failure is not just when an LLM crashes but also when it gives bad output that does not work.

• Three ways to detect breakages include hiring people on mturk to sort through data, asking end-users for feedback, or interpreting user actions.

Comparisons

• Comparing different prompts and versions is a big challenge in production systems.

• Production data is key to knowing which prompt or version works best.

• A/B testing and user segmentation can help provide different prompts to different users.

Optimization

• Choosing which model to use for which task can be challenging.

• Confidence is important when deploying an LLM for thousands or millions of people.

• Front Layer aims to solve these real-world problems by being a reliable platform for building LLM applications.

Prompt Engineering with Prompt Layer

In this section, the speaker introduces Prompt Layer and explains how it can be used to track usage and create prompt templates.

Introduction to Prompt Layer

• Prompt Layer is a tool for tracking usage and creating prompt templates.

• It allows users to see what's going on in their system and track how much they're being charged for prompts.

• Users can organize prompts by creating templates visually and sharing links with co-workers.

The Prompt Registry

• The Prompt Registry is built on top of Langchains prompt template spec.

• It lets users create a template visually by typing in their prompt template.

• Organizing prompts is important because the process of tinkering involves building new versions of existing prompts.

Evaluating Prompts

• Evaluating prompts involves comparing them to each other and checking for breakages.

• The score of a prompt comes from evaluating whether it's better than another prompt.

• Synthetic evaluation is an effective way to evaluate prompts programmatically using few-shot learning.

Analytics Dashboard

• Prompt Layer includes an analytics dashboard that shows which models are being used, latency, and cost.

• The goal of Prompt Layer is to solve problems that exist in real-world prompt engineering by making it reliable.