Why AI Is Incredibly Smart and Shockingly Stupid | Yejin Choi | TED

Artificial Intelligence: Challenges and Solutions

In this section, the speaker introduces the topic of artificial intelligence by referencing a quote from Voltaire and highlighting AI's impressive capabilities alongside its limitations.

The Power and Limitations of Artificial Intelligence

• Despite being a powerful tool, AI faces challenges due to its lack of common sense.

• Large language models in AI show glimpses of artificial general intelligence but often make small mistakes.

• Challenges include the high cost of training extreme-scale AI models, leading to power concentration among tech companies.

Societal Challenges with Extreme-Scale AI Models

This part delves into the societal challenges posed by extreme-scale AI models, focusing on issues related to affordability, power concentration, safety, and environmental impact.

Affordability and Concentration of Power

• Training extreme-scale AI models is expensive, limiting access to only a few tech companies.

• Researchers outside these companies lack means to inspect or dissect these models thoroughly.

Safety Concerns and Environmental Impact

• Questions arise about the safety of AI without robust common sense and whether brute-force scale is the best approach for teaching AI.

• The significant carbon footprint of large language models raises environmental concerns.

Sustainability and Humanistic Approach in AI Development

Here, the focus shifts towards making AI sustainable and humanistic through democratization, safety measures, and ethical considerations.

Democratizing AI and Ensuring Safety

• Proposals are made to make AI smaller for democratization purposes and teach human norms/values for enhanced safety.

The Importance of Common Sense in Artificial Intelligence

This segment emphasizes the critical role of common sense in artificial intelligence development while drawing parallels with dark matter in physics.

Significance of Common Sense

• Common sense is highlighted as a crucial aspect often lacking in current AI systems.

Common Sense in AI

The speaker discusses the evolution of common sense in AI, highlighting the challenges faced and the importance of innovative approaches to enhance AI's understanding.

Evolution of Common Sense

• The speaker reflects on the discouragement faced initially when working on common sense in AI, contrasting it with the current perspective that ChatGPT has almost solved it.

• Emphasizes the need for true common sense in AI as a moonshot goal, suggesting innovation in data and algorithms as an alternative path to extreme scaling.

• Discusses three types of data modern AI is trained on: raw web data, crafted examples, and human judgments, pointing out the limitations of using only raw web data due to biases.

Innovative Data and Algorithms

The speaker delves into the significance of innovative data and algorithms in training modern AI systems effectively.

Training Data Types

• Highlights the importance of using crafted examples and human judgments alongside raw web data to train AI effectively.

• Advocates for transparency in training data by making it open for inspection and correction to align with diverse norms and values.

Teaching Commonsense Knowledge

The focus shifts towards teaching commonsense knowledge to AI through new learning algorithms.

Learning Algorithms

• Contrasts human learning with language models' knowledge acquisition process, emphasizing direct commonsense knowledge acquisition for efficient learning.

• Introduces symbolic knowledge distillation as a potential algorithm to create smaller commonsense models from large language models for better understanding.

Tackling Common Sense Challenges

Addressing the complexities of common sense challenges across various domains within AI research.

Giant Puzzle of Common Sense

• Explores different aspects of common sense such as physical, social, visual common sense, theory of minds, norms, and morals within a broader context.

New Section

In this section, the speaker discusses the importance of finding the right scale rather than assuming that smaller is always better. They emphasize the need for a synthesis of ideas to determine the winning recipe beyond just scale.

Finding the Right Scale

• The speaker mentions that it's crucial to understand that smaller is not always better and that there is likely an optimal scale for success.

• Emphasizes that beyond a certain point of scale, other factors become more critical in determining success.