Ben Goertzel: What AGI Means Now, Open Source AI, and Compute Constraints // AI Inside #106

AI Inside Podcast with Ben Girtzel

Introduction and Sponsorship

• The episode is sponsored by your360.ai, offering a 10% discount through January 2026 with the code "inside."

• Host Jason Howell introduces the episode, highlighting discussions on job replacement due to AI and societal adaptation.

Guest Introduction: Ben Girtzel

• Co-host Jeff Jarvis joins for an interview with Ben Girtzel, founder and CEO of Singularity Net.

• Girtzel is recognized for his work in AGI (Artificial General Intelligence), having coined the term over two decades ago.

The Evolution of AGI Terminology

• Discussion begins on how the meaning of AGI has evolved since its inception.

• Girtzel reflects on how AGI has become a target within the industry, contrasting its original definition with current interpretations.

Origin Story of AGI

• The term originated from a book editing project focused on creating thinking machines that could surpass human intelligence.

• Initial title ideas included "real AI," but concerns about disparaging existing valuable AI applications led to the adoption of "artificial general intelligence."

Defining AGI: Key Concepts

• Girtzel explains the distinction between narrow AI (specific tasks like chess or tax assistance) and AGI (capable of diverse tasks).

• The mathematical definition involves systems that can generalize beyond their experiences and adapt to new environments.

Current Misinterpretations of AGI

• While foundational concepts remain relevant, commercial interests have introduced varied meanings for AGI.

• New definitions often focus on market success or job completion rates rather than true generalization capabilities.

Understanding AGI and Its Evolution

The Core Meaning of AGI

• The core meaning of Artificial General Intelligence (AGI) remains consistent within the research community, despite various ad hoc interpretations introduced by marketers.

Life Extension and General Machines

• Life extension is a term widely recognized, yet it has been diluted with products like "life extension cream," illustrating how terminology can evolve beyond its original intent.

• The discussion shifts to the concept of AI as a general machine, akin to Gutenberg's press, which had limitless potential applications.

Goals in AI Development

• The original goal of AI was to create human-like general intelligence, established in the late 1950s. This contrasts with the later focus on narrow AI due to challenges in achieving that initial vision.

• The divergence towards narrow AI occurred in the 1970s when researchers recognized the complexities involved in building machines that think like humans.

Intelligence Explosion Concept

• J. Good introduced the idea of an "intelligence explosion" in 1965, suggesting that once we achieve truly intelligent machines, they could self-improve indefinitely.

Customization and Future Applications

• A key advantage of AGI is its ability to create specialized narrow AIs for unforeseen tasks, surpassing limitations imposed by human inventiveness.

Value of Generalization

• An AGI capable of customizing its own capabilities or creating new artificial minds holds greater value than a fixed set of narrow AIs.

The Timeline for Achieving AGI

Human-Level vs. Superhuman Intelligence

• Achieving human-level generalization may be arbitrary; however, once reached, it could lead quickly to superhuman capabilities.

Predictions on AGI Development

• Predictions from Kerszwell suggest reaching human-level AGI by 2029 and superintelligence by 2045; however, many now believe this timeline may be too conservative given current advancements.

Current State and Market Influences

• There are varying opinions among industry leaders about timelines for AGI development; some predict earlier dates while others remain cautious about rapid advancements.

Factors Affecting Progression

• While technological understanding provides clarity on possible sequences for development, actual progression depends heavily on market dynamics and funding sources.

Future Predictions: Baby AGI

Upcoming Milestones

• Discussion includes predictions regarding "baby AGI" potentially emerging by early 2025 amidst ongoing debates about timelines for achieving full AGI capabilities.

The Current State of AGI Development

Progress Towards Baby AGI

• The speaker suggests that a "baby AGI" could have been developed by now, but funding has not aligned with this goal.

• They highlight the rapid progress in AI, particularly with math Olympiad-winning capabilities, indicating significant advancements in artificial intelligence.

• The speaker emphasizes that while large language models (LLMs) are not AGI themselves, their integration with other software is bringing us closer to general intelligence.

Funding and Research Dynamics

• Acknowledgment of Patreon supporters who enable the continuation of the show, emphasizing community support for AI discussions.

• Introduction of a sponsor, Your 360 AI, which offers innovative career development tools using voice AI for feedback and coaching.

Career Development Insights

• The lack of effective career development feedback is identified as a primary reason for job dissatisfaction and turnover.

• Your 360 AI provides actionable insights through real conversations about personal growth areas and team dynamics.

LLM Limitations and Market Trends

• Discussion on LLM limitations; experts agree they won't lead to true AGI alone.

• Concerns raised about LLMs consuming too much research funding at the expense of other promising AI paradigms like neuro-symbolic research.

Economic Implications of LLM Dominance

• The speaker speculates that while LLM success may have increased overall funding in AI R&D, it disproportionately diverts resources from alternative approaches.

• They note an increase in absolute funding for various AI efforts since 2021 despite a relative decrease compared to LLM-focused projects.

Risk Aversion in Investment Strategies

• Observations on how businesses tend to favor proven technologies over innovative risks; this trend is more pronounced outside the US.

• The speaker critiques the tendency within capitalism to replicate successful models rather than invest in novel ideas or technologies.

The Future of AI: Beyond LLMs and Towards New Technologies

Current State of AI Investment

• The speaker critiques the disproportionate investment in internal combustion engines over alternative technologies like electric and hydrogen, highlighting a risk-averse mentality in technology funding.

• There is an oversaturation of funding directed towards similar large language models (LLMs), suggesting that the market does not need numerous variations with minor differences.

• The speaker advocates for more investment in diverse AI concepts that have shown promise at smaller scales, rather than just replicating existing models.

Emerging AI Techniques

• Neural symbolic systems, which combine neural networks with logical reasoning, are gaining attention and could play a significant role in future developments.

• Other innovative approaches such as evolutionary programming and genetic algorithms are mentioned as underutilized methods that simulate natural selection to solve complex problems.

• Creativity-enhancing techniques like concept blending are highlighted as valuable but currently overlooked areas within AI research.

Scaling Promising Ideas

• Despite the success of deep neural networks at scale, there remains a lack of investment in scaling other promising AI technologies that have demonstrated potential at smaller levels.

• Investors tend to prefer replicating successful models rather than exploring new methodologies or scaling lesser-known technologies, limiting innovation opportunities.

Opportunities Beyond LLMs

• The speaker emphasizes the vast array of unexplored possibilities within the AI landscape for those willing to innovate beyond LLM replication or application development.

• Over three years, efforts have been focused on building open-source software infrastructure aimed at scaling various AI techniques that mainstream corporate ventures overlook.

Perspectives on AGI and LLM Capabilities

• While acknowledging advancements made by LLMs, the speaker expresses skepticism about their ability to lead to artificial general intelligence (AGI).

• No recent benchmarks or capabilities from LLM developments have significantly challenged this skepticism regarding their path toward AGI.

Historical Context and Predictions

• Historically, predicting which tasks require AGI has proven difficult; many believed complex games like chess were immune to simple algorithmic solutions until they were solved unexpectedly.

• The unpredictability extends to financial markets where straightforward statistical algorithms outperformed human intuition-based trading strategies.

Insights into Human-Like Reasoning

• The speaker notes surprise at how well LLMs can intuit human thinking in nuanced scenarios despite lacking embodied understanding. This capability challenges previous assumptions about what constitutes human-like reasoning.

The Role of LLMs in Programming and Human Jobs

Understanding LLM Capabilities

• The internet provides a vast array of examples for programming and mathematics, making it less surprising that large language models (LLMs) can perform well in these areas.

• A significant takeaway is that LLMs are limited to their training data; however, this limitation may not hinder the majority of human jobs, which often do not require extensive leaps beyond existing knowledge.

• Many jobs could be performed effectively by integrating LLMs with specialized systems without needing artificial general intelligence (AGI), as most tasks involve imitation rather than innovation.

Limitations of LLMs

• A small percentage of human activities necessitate creative leaps beyond training data, which is essential for advancements in fields like science and art—areas where LLMs currently fall short.

• The architecture of LLMs focuses on specific cases from their training data rather than abstracting concepts, limiting their ability to generalize radically.

Open Source vs. Corporate Control in AI Development

The Need for Open Source Models

• There is a call for investment in open-source distributed models to prevent control over AI development from being concentrated within a few corporations.

• Historical examples like Apache and Linux demonstrate the potential success of open-source initiatives, but current needs may differ due to the scale and capital required.

Scale Considerations in AI Development

• A debate highlights differing views on whether achieving advanced AI requires more scale or a different paradigm altogether.

• While scale is important for developing AGI, there are nuanced truths about how much hardware is genuinely necessary compared to what big tech companies suggest.

Investment Strategies for Future AI Technologies

Balancing Scale and Investment

• Historical perspectives suggest that while algorithms can achieve significant outcomes on minimal hardware, modern demands will likely require substantial computational resources.

• Large transformer neural networks exhibit emergent phenomena at higher scales that contribute qualitatively different capabilities compared to smaller models.

Future Demand for Compute Resources

• Despite the possibility of achieving AGI with fewer servers than anticipated, there will be an increased demand across various sectors as AGI becomes integrated into the economy.

Understanding the Future of AGI and Open Source

The Role of Big Tech in AGI Development

• The transition towards Artificial General Intelligence (AGI) is expected to lead big tech companies to create increasingly useful systems, as recognized by leaders like Google founders, DeepMind's Deis and Shane, and Zuckerberg.

• There is a nuanced understanding among these tech leaders that contrasts with media portrayals.

Open Source Terminology and Its Implications

• Sam Altman from OpenAI has criticized the organization for being on "the wrong side of history" regarding open source initiatives.

• The terminology surrounding open source has been historically complex, with debates over what constitutes true openness (e.g., GPL vs. Apache/MIT licenses).

Challenges in AI Openness

• Simply opening software code does not fulfill the spirit of open source if training data and infrastructure details are not also shared.

• To achieve genuine openness, it’s essential to provide not just code but also explanations about data sources and tools for users to replicate or modify systems.

Economic Considerations in AI Infrastructure

• Current AI methods often require massive amounts of data and compute resources, which creates barriers for smaller entities competing against big tech.

• Big tech benefits from promoting AI methods that necessitate large-scale infrastructure, limiting competition from those who could innovate with less resource-intensive approaches.

Vision for Decentralized AI Development

• There is a need to develop AI methods that can operate on minimal data and diverse computing infrastructures to foster broader participation in AGI development.

• Projects like OpenCog Hyperon aim to build flexible neural symbolic evolutionary AI infrastructures beyond traditional deep learning models.

Exploring Decentralization in AGI

• While some aspects of AGI may require centralized server farms, there is potential for decentralized networks (like mesh networks or personal devices).

• Investigating how much functionality can be decentralized will help minimize reliance on extensive server farms while still achieving effective AGI solutions.

Interview with Dr. Girtzil: The Future of Open AI

Defining Openness in AI

• Dr. Girtzil emphasizes the importance of a clear definition of "open" in the context of AI, expressing concerns about large companies and government regulations that may hinder open development.

Key Stakeholders to Influence

• To realize his vision for open AI, Dr. Girtzil identifies three critical groups to influence:

• Governments must avoid passing regulations that favor big tech monopolies.

• The research and development (R&D) community should resist easy-to-use tools from big tech and instead engage with more raw, decentralized options.

Community Engagement and Development Tools

• He highlights the need for R&D communities to embrace less polished tools that promote open development, drawing parallels with Linux's success despite its steeper learning curve.

• Linux is cited as a model for how an open foundation can lead to widespread adoption across various political landscapes.

Funding Requirements for Open AI Initiatives

• While developing open and decentralized solutions requires less funding than centralized approaches, significant investment is still necessary—estimated at hundreds of millions for hardware.

• Dr. Girtzil shares his experience funding AGI efforts through cryptocurrency, noting both benefits and distractions from this approach.

Challenges Ahead

• There remains uncertainty regarding whether the crypto community will provide sufficient funding for the open decentralized AI movement.

• Ultimately, he stresses the necessity of aligning regulatory support, community engagement, and adequate funding to foster an environment conducive to open AI development.

Advice for Future Generations

• In response to questions about education in technology and society, Dr. Girtzil advises young people to pursue their passions while also being adaptable learners prepared for rapid changes in job markets due to technological advancements.

The Future of Work and AGI

The Risks of Settling into Ruts

• Avoid becoming complacent in any career path, as it can lead to disruption. Emphasize the importance of adaptability in a rapidly changing job market.

Importance of Foundational Knowledge

• Advocate for learning fundamental subjects such as mathematics, physics, literature, and philosophy. These basics remain crucial for understanding nature and human experience.

Changing Career Landscape

• The notion that choosing a specific major guarantees job security is outdated. Traditional paths like plumbing or electrical work may not be as reliable due to technological advancements.

Concerns About Early Stage AGI

• There are worries about the transition period between early stage AGI and superintelligence, particularly regarding economic stability and job displacement in developing countries.

Ethical Dilemmas During Transition Period

• As jobs are automated by AI, there is concern over how people in developing nations will cope without access to basic necessities. This raises ethical questions about minimizing suffering during this interim phase before superintelligence arrives.

Predictions on AGI Development Timeline

• Speculation suggests that human-level AGI could emerge around 2029. However, the implications of what happens immediately after its introduction remain uncertain and require careful consideration.

Opportunities Before Early Stage AGI Arrival

• Despite impending changes from early stage AGI, there are still numerous opportunities for innovation and business growth by identifying niches that will be transformed by emerging technologies.

Excitement Around Technological Advancements

• Current advancements in AI allow for rapid development from concept to realization. This era presents unique opportunities to implement ideas from decades of research at scale.

Acknowledgment of Contributions

• Gratitude expressed towards Ben Girtzel for his insights on AGI, highlighting his role in coining relevant terminology within the field.

Closing Remarks on Support and Resources

• Encouragement to explore resources related to the show "AI Inside" while expressing appreciation for contributions made by guests like Jeff Jarvis.

Giveaways and Community Engagement

Introduction to Giveaways

• The host discusses the benefits of supporting the show through Patreon, including ad-free episodes, access to a Discord community, and occasional giveaways.

Items for Giveaway

• Three items are announced for giveaway: a Google Found It hat, an AI Mode t-shirt, and a Google DeepMind t-shirt. The host expresses some reluctance about giving away the AI Mode t-shirt but confirms it is too late to change that decision.

Announcement of Winners

• The first winner is revealed as Christian Blazer (the host's nephew), who will receive the Google hat. Robert Frisky wins the AI Mode t-shirt, while Tom Roughly receives the DeepMind t-shirt.

Gratitude Towards Supporters

• The host emphasizes gratitude towards patrons for their support throughout the year, stating that their contributions are essential for maintaining the show's operations.

Future Communication with Winners

• The winners will be contacted via direct message or text regarding shipping details after Christmas. The host mentions additional swag available for executive producers and lists current supporters by name.