Interactive Simulacra of Human Opinions and Behavior

Introduction to the Workshop

The speaker welcomes everyone to the Cyber Policy Center Workshop and introduces Professor Michael Bernstein from the computer science department. The workshop will focus on interactive simulacra of human opinions and behavior.

Workshop Introduction

• The speaker welcomes everyone to the Cyber Policy Center Workshop.

• Professor Michael Bernstein from the computer science department will be speaking about interactive simulacra of human opinions and behavior.

• The audience is encouraged to Google any unfamiliar terms in the title.

• Questions can be submitted through Zoom's Q&A feature.

Opening Remarks

The speaker provides some opening remarks before introducing Professor Michael Bernstein's talk.

Opening Remarks

• The speaker thanks everyone for attending.

• As a social computing designer, Professor Bernstein is interested in building technologies that bring people together online.

• The talk will focus on interventions that can address challenges in AI and its intersection with social and societal issues.

• Questions from the audience can be submitted through Zoom's Q&A feature.

Challenges in AI Design

Professor Bernstein discusses how AI has been designed as a separate entity without considering its impact on society. He highlights the need to consider whose voices are represented by AI systems.

Challenges in AI Design

• AI has been treated as a hermetically sealed box, but it is important to consider who benefits or is negatively impacted by it.

• Progress has been made in addressing societal impacts of AI, but there are still questions about whose voices are represented by AI systems.

• It is crucial to understand how AI is embedded within our social contexts and how it interacts with us.

Projects Exploring AI Integration

Professor Bernstein introduces two projects that explore how AI can integrate with societal and social contexts. The first project focuses on new metaphors for AI in social media, while the second project explores generative agents.

Project 1: New Metaphors for AI in Social Media

• This project aims to develop new metaphors for AI in social media contexts.

• It explores how AI can be used to understand and shape social norms and interactions.

• The project is of personal interest to Professor Bernstein and others in the room.

Project 2: Generative Agents

• This project focuses on generative agents that can empower interactive applications.

• These agents aim to teach us about social norms and interaction.

Jury Learning - Addressing Societal Disagreements

Professor Bernstein discusses "jury learning," a project led by his PhD student Mitchell Gordon. The project addresses the use of AI classifiers in spaces with intense societal disagreements.

Jury Learning - Addressing Societal Disagreements

• AI classifiers are widely used in social environments for tasks like toxicity detection and misinformation detection.

• The Google Perspective API is an example of an AI system that provides probabilities for text toxicity.

• Determining whether certain content should be removed depends on societal opinions, which are gathered through annotators' input.

• Ground truth labels are created by aggregating annotators' opinions, often using majority voting.

Challenges in Determining Toxicity

Professor Bernstein discusses the challenges of determining toxicity based on societal opinions when training AI models.

Challenges in Determining Toxicity

• Gathering data for training AI models involves asking multiple annotators their opinions on whether certain content should be considered toxic or not.

• Annotators may have different perspectives, leading to disagreements on what should be considered toxic.

• Majority voting is often used to determine the ground truth label, but it may not always accurately represent societal opinions.

The transcript continues beyond this point, but the provided content covers the main topics discussed in the given timestamps.

The Challenge of Disagreement in Machine Learning

In this section, the speaker discusses the challenge of disagreement in machine learning tasks and how it can affect the accuracy of AI algorithms.

Disagreement in Machine Learning

• Machine learning algorithms are trained using labeled examples to determine the correct answer or behavior.

• However, even experts may disagree on what the right answer should be for certain tasks.

• This is particularly evident in tasks like toxicity detection, where there can be significant disagreement on whether a comment is toxic or not.

• Majority voting, commonly used in data annotation and machine learning, tends to silence minoritized groups whose opinions may differ from the majority.

Widespread Disagreement

• Disagreements are prevalent across various domains. For example:

• In a large-scale dataset used for toxicity labeling, 40% of annotators disagreed on average with anything labeled as toxic.

• Reddit moderators disagree about one-third of the time on what content should be removed.

• Fact checkers also exhibit disagreement when evaluating articles for false information.

Importance of Addressing Disagreement

• Disagreements matter not only in social media but also in fields like medicine and design where different perspectives exist.

• To address this challenge, a new approach called "jury learning" is proposed.

Jury Learning: A New Approach

This section introduces the concept of jury learning as an alternative approach to traditional machine learning methods.

Introducing Jury Learning

• Jury learning changes the metaphor of how AI operates by considering a group of people as a jury.

• The composition of this jury represents different intersectional identities or characteristics that influence their perspectives.

• The goal is to determine what proportion of this jury agrees on certain decisions to shape AI's behavior.

Modeling Individuals Instead of Aggregates

• Instead of predicting the aggregate outcome, jury learning focuses on predicting what each individual juror would say.

• AI is used to predict the opinions of each juror based on their characteristics and perspectives.

• These individual opinions are then aggregated to form a collective decision.

Implementing Jury Learning at Scale

• The composition of the jury can be defined by system designers, considering characteristics that represent diverse perspectives.

• Random sampling is used to select jurors from groups that match specific characteristics.

• This approach allows for modeling individuals' opinions and aggregating them into a collective decision.

Conclusion

Jury learning offers a new approach to address the challenge of disagreement in machine learning tasks. By considering diverse perspectives and modeling individual opinions, AI algorithms can better reflect different viewpoints. This approach has the potential to improve fairness and accuracy in various domains where disagreements exist.

Understanding AI Modeling and Simulating

In this section, the speaker discusses the ease of modeling and simulating different scenarios compared to real-life situations. They explain how multiple juries can be sampled to get a sense of their opinions and visualize the likelihood of certain results.

Modeling and Simulating Juries

• When modeling and simulating, it is easy to sample many different juries.

• Sampling multiple juries helps in understanding different perspectives and opinions.

• Visualizations can provide a straightforward interpretation of what the AI is suggesting.

• A median of means estimator is used to determine the value for each jury's opinion.

Sense Making and Explainability in AI Decisions

This section focuses on sense making and explainability in AI decisions. The speaker explains how disagreements within identity groups can exist, highlighting the importance of visualizing these differences for better understanding.

Disagreements Within Identity Groups

• Different instantiations of juries show disagreement even within identity groups.

• Visualizations help in understanding these disagreements among sampled people.

• It provides a notion of sense making and explainability in AI decisions.

Unpacking AI Predictions for Juror Votes

The speaker briefly explains how AI predicts each juror's vote using a data set with labeled inputs from people. They mention collaborative filtering as an approach to observe patterns between how people vote.

Predicting Juror Votes

• Data sets with labeled inputs are used to predict juror votes.

• Collaborative filtering helps identify patterns between people's voting behavior.

• Similarity between individuals' votes influences predictions for new inputs.

Addressing Bias in Jury Selection

The speaker discusses the importance of addressing bias in jury selection and suggests focusing on selecting and training appropriate voices rather than relying solely on fairness debiasing post-talk.

Bias in Jury Selection

• Jurors can be biased, but addressing bias should start upstream by considering whose labels are being listened to.

• Selecting and training appropriate voices can solve representation and understandability issues effectively.

• AI fairness is important, but transparency and explicitness in jury selection can have a significant impact.

Implicit Juries and Transparency

This section highlights the existence of hidden and implicit juries in every data set used for AI. The speaker emphasizes the need to make these juries transparent and explicit instead of remaining implicit.

Hidden Juries in Data Sets

• Every data set used for AI already has a hidden and implicit jury.

• Making these juries transparent and explicit is crucial.

• Tatsu Hashimoto's approach helps back solve for the existing juries.

Impact of Using Author-Created Juries

The speaker presents the impact of using author-created juries by recruiting moderators from online platforms. They measure whether jury learning represents more diverse opinions compared to state-of-the-art classifiers.

Impact of Author-Created Juries

• Moderators from online platforms were recruited to create juries for toxicity tasks.

• Author-created juries aim to represent more diverse opinions than baseline classifiers.

• Graphs show proportional representations of different groups' opinions in baseline data sets versus author-created juries.

Changes in Classifier Behavior

This section discusses the impact of changing certain factors on classifier behavior, such as police suicide, LGBTQ+ issues, and mental illness.

Material changes in classifier behavior

• About 14% of classifications end up changing compared to the current state-of-the-art classifier.

• Changing factors like police suicide, LGBTQ+ issues, and mental illness can lead to these changes.

Uncontroversial and controversial factors

• Some factors are uncontroversially fine or universally agreed upon as problematic (e.g., racial ethnic slurs).

• Other factors may be more controversial and require further discussion.

Redesigning Existing Systems

This section explores the possibility of redesigning existing systems and speculates on how future AI systems might behave.

Simulating believable human behavior

• Various applications have been envisioned that simulate believable human behavior.

• Examples include simulating social systems, cognitive models, virtual worlds, social robots, and interactive training.

Challenges in modeling human behavior

• Human behavior is complex and contingent, making it difficult to effectively model.

• Previous attempts using large datasets or language models have had limited success.

Creating Interactive Opportunities

This section discusses the potential for AI systems to learn about social interaction and create new interactive opportunities.

Generative agents

• The concept of generative agents is introduced.

• These agents can simulate believable human behavior by drawing inferences about themselves, others, their environment, creating plans based on characteristics and experiences.

Applications of Generative Agents

This section explores various applications of generative agents in simulating human behavior.

Wide variety of traits and personalities

• Language models like Chat GPT can be prompted to take on different backgrounds, experiences, and traits.

• This allows for the exploration of different personalities and their potential actions in various situations.

Challenges in believability

• Simply prompting Chat GPT is not enough to achieve believable behavior.

• Believable behavior requires accounting for social dynamics, relationships, and learning over time.

Complexities of Human Behavior

This section highlights the challenges in modeling complex human behavior.

Resilient complexity

• Human behavior is resiliently complex and cannot be easily modeled.

• Previous attempts using large datasets or language models have had limited success.

Large Language Models (LLMs)

This section discusses recent advancements in large language models and their potential applications.

Versatility of LLMs

• LLMs like Chat GPT can be prompted to take on a wide variety of backgrounds, experiences, and traits.

• They can simulate different agents with unique personalities and explore their potential actions in various situations.

Challenges in Simulating Believable Behavior

This section addresses the challenges faced when simulating believable behavior using AI systems.

Limitations of simple prompts

• Simple prompts may result in agents that only provide superficial responses without considering social dynamics or relationships.

• To achieve believable behavior, AI systems need to account for learning, growth of friendships, grudges, and other social dynamics.

Generative Agents in a Video Game World

This section introduces generative agents within a video game world as a testbed for simulating believable human behavior.

Smallville: A simulated town

• The video game world called Smallville is introduced as a testbed for generative agents.

• It consists of 25 generative agents that simulate believable human behavior by drawing on generative AI modules.

Simulating Believable Human Behavior

This section explores the capabilities of generative agents in simulating believable human behavior.

Inferences and plans

• Generative agents draw inferences about themselves, others, and their environment.

• They create plans based on their characteristics, experiences, and goals.

A Simulated World to Explore

This section highlights the simulated world created by generative agents for exploring believable human behavior.

A simulated world with independent agents

• Each generative agent acts independently, living its life with daily routines and interactions.

• The agents have their own interests, experiences, and plans that shape their behavior within the simulated world.

Creating a Student Athlete Agent

The speaker discusses the creation of a student athlete agent and how changing its description can affect its behavior.

Creating the Agent

• A student athlete agent is created with a morning routine consisting of activities like waking up, brushing teeth, going for a run, cooking breakfast, and heading to class.

• Changing the agent's description will result in changes in its behavior.

Keeping a Database of Experiences

The speaker explains how they maintain a large database of an agent's experiences and use relevant subsets to determine their behavior.

Database Management

• A large database is maintained to store all the agent's experiences.

• Over time, relevant subsets are pulled from the database to determine the actions and behaviors based on given situations.

Gathering Experiences and Building Inferences

The speaker describes the process of gathering experiences, reflecting on them, building higher-level inferences about oneself and others, and planning future actions.

Loop of Actions

• The process involves gathering experiences and memories.

• Reflection on these experiences leads to building higher-level inferences about oneself and others.

• Planning future actions based on these reflections completes the loop.

Implementing in a Game Village Setting

The speaker explains how they implemented this concept into a small game village setting using JavaScript-based video game creation software called Phaser.

Game Village Setup

• A small game village was created using Phaser.

• The village includes various locations such as dorms, college buildings, grocery stores, houses with different rooms (bathrooms, kitchens, common rooms), gardens, bookshelves, and tables.

• Each agent in the game is defined with their own characteristics and initial memories.

Agent Interactions through Natural Language Dialogue

The speaker discusses how agents interact with each other primarily through natural language dialogue.

Natural Language Dialogue

• Agents engage in conversations using natural language dialogue.

• Example conversation: Isabella and Sam discussing the upcoming mayoral election.

Initial Memories and Interactions

The speaker explains how each agent has an initial memory that defines their identity, relationships, interests, and interactions with others.

Initial Memory Setup

• Each agent has an initial memory that includes information about their identity, relationships, occupation (e.g., John Lin works in a pharmacy), and interests.

• Agents interact with each other based on their memories and relationships.

Interaction with Game World

The speaker describes how agents interact with the game world through text outputs translated into concrete movements.

Text Outputs to Movements

• Agents' text outputs are translated into concrete movements within the game world.

• Users can control agents' actions through dialogue or by manipulating the game world.

• Influencing agent behavior is possible by assuming the role of another agent or manipulating the environment.

Morning Routine of Lynn Family

The speaker provides an example of the morning routine of the Lynn family within the game village setting.

Lynn Family's Morning Routine

• The Lynn family follows a morning routine consisting of conversations, packing, getting ready for work/school, etc.

• Example interaction between John (father) and Eddie (son) about Eddie's music theory class project.

Isabella's Valentine's Day Party

In this section, Isabella plans and prepares for a Valentine's Day party, inviting friends and customers. Another character, Klaus, has a crush on someone and plans to attend the party with them.

Isabella's Party Preparation

• Isabella organizes a Valentine's Day party the day after sharing her plans with others.

• She spends the afternoon decorating for the party.

• Isabella enlists another agent to help her decorate.

Klaus' Crush and Party Attendance

• Klaus has a crush on someone and intends to go to the party with them.

How Memory Retrieval Works

This section explains how memory retrieval works in language models and highlights three important characteristics: recency, importance, and relevance.

Challenges of Recording Complete Memory

• Recording everything that happens can overwhelm language models.

• Language models have limited input capacity.

Three Characteristics of Memory Retrieval

1. Recency:

• Memories become less relevant over time.

• Recent events are given more weight in memory retrieval.

2. Importance:

• Agents assign importance to memories based on personal significance.

• Memories related to significant events hold more weight in retrieval.

3. Relevance:

• Memories are retrieved based on their relevance to the current context.

• Similar memories are retrieved when they align with the current situation or task at hand.

Teaching Language Models Reflection

• To enable generalization and inference capabilities, language models need to be taught reflection.

• The "reflect Loop" helps extract higher-level insights from past experiences.

• These insights form a tree-like structure of reflections that contribute to an agent's understanding of itself.

Planning Behavior in Agents

This section discusses how agents plan their behavior by considering their current status, past experiences, and future intentions.

Path through the World

• Agents need a coherent path through the world to make sense of their actions.

• Planning involves creating a high-level sketch of an agent's day and diving into more detail for specific time intervals.

Reacting to New Information

• When agents encounter new information or events, they consult the language model to determine appropriate reactions.

• The language model considers the description of the agent and recent events to provide guidance on how the agent would react.

Evaluating Agent Believability

This section explains how agents' believability is evaluated using interviews and role-playing exercises with both human evaluators and the full architecture.

Interviewing Agents

• Agents are interviewed using structured questions about their daily schedule, memory, planning, and higher-level reflections.

• The responses generated by the language model during interviews are evaluated for coherence and accuracy.

Role-playing Exercises

• Human evaluators watch agents' behavior and role-play as individual characters.

• Other evaluators rank the responses from different parts of the architecture to assess believability.

The transcript provided does not include timestamps beyond 1855 seconds.

Understanding True Skill and Generalization

The speaker introduces the concept of a ranking system called ELO, which is used to determine skill levels in games like chess. They explain that true skill is a generalization of this system and how Xbox has implemented it across multiple games.

True Skill and ELO Ranking System

• The ELO ranking system is used to determine skill levels in games.

• True skill is a generalization of the ELO system.

• Xbox has implemented true skill across multiple games.

Contributions to Agent Behavior Believability

The speaker discusses how different components of the architecture contribute to the believability of an agent's behavior. They explain that observation, planning, and reflection are key factors in enhancing believability.

Components Contributing to Believable Behavior

• Observation, planning, and reflection are important components of the architecture.

• These components contribute to the believability of an agent's behavior.

• Removing any of these components significantly impacts believability.

Errors in Memory Retrieval

The speaker talks about errors that can occur when agents fail to retrieve memories accurately. They give an example where an agent fails to recall information about a local election but had actually been told about it before.

Errors in Memory Retrieval

• Agents can fail to retrieve memories accurately.

• Example: Rajiv didn't remember information about a local election despite being informed previously.

• Agents may also hallucinate or invent things that didn't happen based on their memory stream.

Emergent Social Dynamics

The speaker explains their interest in understanding emergent social dynamics using a simulation. They describe running a simulation with 25 characters for two game days to observe if it leads to a civil war or other outcomes.

Studying Emergent Social Dynamics

• A simulation with 25 characters was run for two game days.

• The goal was to observe emergent social dynamics.

• The initial agent descriptions included specific goals and relationships.

• Information started to diffuse among the agents, leading to interactions and social dynamics.

Planning a Valentine's Day Party

The speaker discusses the outcome of planning a Valentine's Day party within the simulation. They highlight errors and considerations when deploying these systems, such as agents being overly compliant or formal in their dialogue.

Outcome of Planning a Party

• 12 agents heard about the Valentine's Day party, but only 4 showed interest.

• Rajiv declined due to focusing on an upcoming show.

• Agents can be overly compliant and suggest activities like Shakespearean readings without receiving clear rejection from the host.

Ethical Considerations and Societal Impact

The speaker addresses ethical concerns and potential risks associated with AI chatbots. They mention parasocial relationships, deep fakes, misinformation generation, tailored persuasion, and the importance of aligning models with human values.

Ethical Considerations and Risks

• Potential risks include parasocial relationships formed with AI chatbots.

• Mitigation involves explicit disclosure of computational nature and value alignment by developers.

• Other risks include deep fakes, misinformation generation, and tailored persuasion techniques.

• Developing technologies should consider societal impact and ethical implications.

This summary provides an overview of the main topics discussed in the transcript. For more detailed information, please refer to the corresponding timestamps provided.

The Role of AI Agents in Community Engagement

In this section, the speaker discusses the use of AI agents as a substitute for human or community involvement. They emphasize that while AI agents can be used as a complement to community engagement, they should not be seen as a complete replacement.

AI Agents as Complements, Not Substitutes

• Syntheticusers.com has pitched the idea of using AI chatbots to interview and gather information from communities without human interaction. However, the speaker emphasizes that AI agents should not replace human or community involvement.

• While there may be situations where using AI agents as a complement to community engagement is beneficial, completely avoiding direct communication with the community can lead to problems.

Less Anonymity and Unexpected Attention

This section highlights two points: the unexpected attention received after putting their research on archive and the realization that AI-generated content is less anonymous than expected.

Unexpected Attention and Archive Placement

• After putting their research on archive while it was under review, it gained attention from AI Twitter and generated significant interest.

• The speaker acknowledges June, the lead grad student who received attention for their work.

Less Anonymity in AI-generated Content

• The speaker mentions that they were surprised by how less anonymous AI-generated content turned out to be.

• They note that due to this discovery, they are cautious about using such technology without considering its implications.

Questions and Discussion

In this section, questions are raised by participants regarding combining jury learning with simulated jury deliberation and addressing believability in generative characters' embellished memories through hallucination.

Combining Jury Learning with Simulated Deliberation

• A participant suggests combining jury learning with simulated jury deliberation.

• The speaker acknowledges progress made in this area but mentions that they have not published anything yet.

• Two types of deliberation are considered: generative text-based debates and observing the impact of counterarguments on individuals' positions.

Addressing Believability in Embellished Memories

• A participant raises a question about the believability of generative characters' embellished memories through hallucination.

• The speaker explains that during evaluation, agents' believability decreases if they provide false information.

• While embellishment can increase believability in certain contexts, it is viewed as a dangerous component in generative models due to potential risks.

• The speaker acknowledges the need for control over narrative consistency, especially in applications like video games.

Progress and Future Directions

This section highlights ongoing progress and future directions related to simulated jury deliberation and other research areas.

Simulated Jury Deliberation

• The speaker confirms their interest in simulated jury deliberation but mentions that more extensive data sets are required for effective implementation.

• A graduate student working on this topic found something else interesting to discuss, temporarily pausing progress on simulated jury deliberation.

Openness to Discussion

• The speaker expresses openness to discussing various topics related to their research and invites further conversation.

Timestamps may vary slightly depending on the source.

New Section

In this section, the speaker discusses the use of generative agents as simulations for studies and the potential benefits and limitations of using them.

Using Generative Agents for Simulations

• The outcomes of complex emergent systems are underdetermined and can go in many different ways. Sociology has shown that it is challenging to predict exact outcomes.

• Some behavioral scientists are interested in using generative agents as simulations for their studies. This approach could be useful for studying specific scenarios or fabrications.

• If the material being studied is related to the generative agent's behavior, it could provide valuable insights. However, if it is unrelated, it may produce unrealistic social behaviors that undermine the study's objectives.

• The idea of creating representative simulations to predict how people will respond to interventions or situations like a pandemic is intriguing but still in its early stages.

• The next stage would involve inputting more personalized information into these simulations, such as individual characteristics and behaviors, which could be beneficial for various fields like political consulting or video game development.

• There is an ongoing debate about how well generative models replicate human psychology. Current understanding lacks concrete knowledge about when these models work effectively and when they deviate from reality.

New Section

In this section, the speaker explores the concept of representation in virtual worlds and how it relates to different types of decisions and juries.

Representation in Virtual Worlds

• Representation matters in virtual worlds, just like in real life. However, determining what constitutes appropriate representation can vary depending on the context.

• Different types of decisions may require different juries with specific expertise or backgrounds. For example, disinformation cases might need a different jury composition compared to cases involving child exploitation or toxicity.

• Conditional juries can be implemented in virtual worlds where the jury composition is tailored to the specific topic or issue being addressed.

• The desired representation in virtual worlds is a broader discussion happening across various fields, including media, movies, and animation.

The transcript provided does not contain enough content for additional sections.

New Section

In this section, the speaker discusses the importance of explicitly imagining and being forward-thinking when considering what something should look like. They also address the question of whether complete randomization or conditional randomization is achievable and desirable in real-life AI applications.

Imagining What Something Should Look Like

• When asked to imagine what something should look like and get explicit about it, being forward-thinking is more likely compared to not thinking about it at all.

Randomization in AI Applications

• The speaker explores the possibility of complete randomization or conditional randomization in the panel composition of jewelry.

• They discuss whether real-life AI applications would try to signal their use of randomization patterns to enhance trustworthiness.

• The concept of achieving a reasonably randomized jury composition, similar to a gold standard, is mentioned.

New Section

In this section, the speaker delves into the practical aspects and potential responses to the identified problem related to jury selection in AI applications.

Internal Use and Disclosure by Tech Companies

• The speaker raises two separate questions regarding tech companies:

• Would a tech company internally debate and use methods such as conditional randomization without disclosing it?

• If they did use such methods, would they disclose the composition of their jury?

• It is suggested that using conditional randomization can actually improve performance in certain tasks, such as building a perspective API toxicity classifier.

Normative Slant and Participatory Approaches

• There is a discussion on whether tech companies would disclose their use of conditional randomization. The speaker expresses a more cynical view on this matter.

• The speaker mentions that there is a normative slant to the jury learning paper, suggesting that disclosure should be encouraged.

• They highlight the importance of participatory approaches and how they can allow for deliberation and contestation, leading to qualitatively different outcomes compared to just looking at precision-recall curves.