제프리 힌턴이 말하는 AI의 영향력과 잠재력

Introduction

In this section, the speaker talks about the current state of AI and machine learning.

The Pivotal Moment in AI

• The speaker describes the current moment in AI as a pivotal one.

• GPT has shown that big language models can do amazing things.

• The general public has suddenly become aware of this due to Microsoft's release.

Chat GPT and Early Language Models

In this section, the speaker discusses their experience with early language models and their thoughts on Chat GPT.

Experience with Early Language Models

• The speaker has used many things similar to Chat GPT before.

• They were amazed by an earlier model at Google that could explain why a joke was funny.

Thoughts on Chat GPT

• Chat GPT itself did not amaze the speaker much.

• However, they were surprised by the public's reaction to it.

Two Schools of Thought in AI

In this section, the speaker talks about two schools of thought in AI and how they differ from each other.

Mainstream AI vs Neural Nets

• There were two schools of thought in AI: mainstream AI and neural nets.

• Mainstream AI based its theories on reasoning and logic while neural nets focused on studying biology.

Neural Nets' Approach to Learning

• Neural nets believed that connections between neurons change, which is how you learn.

• This approach eventually proved successful despite being initially dismissed by mainstream AI.

Convincing People About Neural Networks

In this section, the speaker talks about whether there was anything they could have said back then to convince people about neural networks.

Why Neural Networks Weren't Working in the 1980s

• The big issue in the 1980s was whether a big neural network with lots of neurons could learn by just changing the strengths of connections.

• Mainstream AI thought this was ridiculous.

Convincing People About Neural Networks

• The speaker believes that they could have said that neural networks weren't working well in the 1980s because computers weren't fast enough and data sets weren't big enough.

• However, this wouldn't have convinced people at the time.

Understanding How the Brain Works

In this section, the speaker talks about their core interest in understanding how the brain works.

Interest in Understanding How the Brain Works

• The speaker's core interest is not in creating AI but rather in understanding how the brain works.

• They believe there is currently a divergence between artificial neural networks and how the brain actually works.

Back Propagation Technique

• All of the big models now use a technique called back propagation which was popularized by the speaker in the 80s.

• However, they don't think this is what the brain is doing.

Fundamental Difference Between Two Paths to Intelligence

In this section, the speaker talks about two different paths to intelligence and why they differ from each other.

Two Different Paths to Intelligence

• There are two different paths to intelligence: biological path and digital path.

• Biological path involves hardware that's analog while digital path involves hardware that's digital.

Back Propagation vs Hardware Learning

• Back propagation is used for digital path while hardware learning is used for biological path.

• The speaker believes that back propagation isn't what the brain is doing.

Communicating with Computers

In this section, the speaker discusses how current computer models are able to communicate and process data at an almost infinite level due to their ability to run on clones of the same model on different computers.

The Power of Computer Models

• Current computer models can communicate and process data at an almost infinite level.

• These models are clones of the same model running on different computers, allowing them to see huge amounts of data.

• Chat GPT is an example of a language model that knows much more than any one person. It can write poems and do well in bar trivia competitions.

Neural Networks in the Past

• In 1986, the speaker published a language model that predicted the last word in a sentence. However, it was not taken seriously because it was trained on a small dataset.

• Back then, computers were not powerful enough for neural networks to be effective.

Deep Learning

• Around 2006, deep learning began to take off as better ways of initializing networks were discovered.

• By 2009, something had already been produced using these techniques.

Introduction to Deep Neural Nets

In this section, the speaker introduces deep neural nets and discusses two significant events that occurred in 2012.

Deep Neural Nets

• Deep neural nets are a type of machine learning algorithm that can learn from large amounts of data.

• They consist of multiple layers of interconnected nodes that process information and extract features.

• These algorithms have been used for various applications such as speech recognition and object recognition.

Significant Events in 2012

• Google deployed deep neural nets for speech recognition on Android, which led to significant improvements in accuracy.

• Two students developed an object recognition system that outperformed previous systems by using a large database of images and feature detectors at different levels.

Object Recognition System

In this section, the speaker explains how their team's object recognition system worked.

How the System Worked

• The system used a large database of images with a million images from a thousand different categories.

• The goal was to identify the primary object in an image with minimal errors.

• Feature detectors were created at different levels to detect edges, angles, circles, beaks, eyes, etc., which were then combined to recognize objects like birds.

Backpropagation Algorithm

In this section, the speaker explains how backpropagation works.

Backpropagation Algorithm

• Backpropagation is an algorithm used to train deep neural nets by adjusting connection strengths between nodes based on prediction errors.

• It involves calculating the error or discrepancy between predicted output and actual output and then propagating it backward through the network to adjust connection strengths accordingly.

Understanding Neural Networks

In this section, the speaker explains how neural networks work and why they are better than traditional approaches.

Symbolic AI vs. Neural Networks

• Symbolic AI is an approach that manipulates symbols to recognize things like images.

• Neural networks work much better than symbolic AI for recognizing complex images.

• The only place where symbols are used in neural networks is at the input and output.

Teaching People to Code

• It may not make sense to teach people to code in the future as computers become more advanced.

• Computers are now comparable with radiologists at recognizing medical images, but it may still be worth having coders for a while.

Investing in Coherent.ai

In this section, the speaker talks about investing in Coherent.ai and why he believes they will be successful.

Why Invest in Coherent.ai?

• Coherent.ai takes big language models developed by companies like Google and makes them available to other companies.

• They have a significant lead in this area, which makes them valuable.

Biological Route to Intelligence

In this section, the speaker discusses the possibility of creating a new kind of computer that mimics biological intelligence.

Mimicking Biological Intelligence

• A new kind of computer could mimic biological intelligence by allowing each brain to be different.

• This would require communicating knowledge from one brain to another using a common language.

The Power Consumption of Digital Computers vs. Biological Brains

In this section, the speaker discusses the power consumption of digital computers and biological brains.

Digital Computers vs. Biological Brains

• Big language models are sharing connection strengths like how one bird can learn to recognize cats and another bird can learn to recognize birds, and they share their connection strengths.

• Digital computers have to do identical things, but you can't make different biological brains behave identically so you can't share connections.

• Running a digital computer requires high power, whereas running a brain requires much lower power. A low-power system allows for noise and adaptation to that particular system's noise.

• The brain runs on 30 Watts while big AI systems need a megawatt because they have lots of copies of the same thing.

• There will be a phase when we train on digital computers but run it on very low-power systems once something is trained.

Impact of AI Technology on People's Lives

In this section, the speaker talks about how AI technology will impact people's lives.

Future Applications of AI Technology

• It's hard to pick one thing that AI technology will do that will impact people's lives because it will be everywhere.

• Google uses big neural nets to help decide what's best for search results. Chat GPT is already getting everywhere.

• Chat GPT doesn't understand truth as it is being trained on inconsistent data trying to predict what someone might say next on the web where people have different opinions.

• Systems will move towards understanding different world views and providing answers based on those world views.

• It's good to have a consistent world view if you want to act in the world, but people have their own truths.

• There is a big governance challenge in deciding what's true. We don't want some big for-profit company controlling how we turn the neurons.

• Google is careful not to control what's true at present. They refer users to relevant documents with all sorts of opinions in them.

• Microsoft is less careful than Google when it comes to chatbots. Google will probably come with lots of warnings that it's just a chatbot and not necessarily believe everything it says.

Limitations of Chatbots

In this section, the speaker discusses the limitations of chatbots.

Limitations of Chatbots

• Some things are offensive and can't be said by chatbots, so companies meddle with them so they won't say offensive things.

• There are limits to what can be done that way because there will always be things that weren't thought of.

• Google will be far more careful than Microsoft when releasing its chatbot and will probably come with lots of warnings.

The Challenges of AI

In this section, Elon Musk discusses the challenges of AI and how it could potentially be dangerous to humanity.

Concerns about AI

• There is a big open issue regarding the regulation of AI.

• Language models will have to understand that there are different points of view and that completions make relative to a point of view.

• Some people worry that AI could take off very quickly and we might not be ready for it.

• Musk thought it would be 20 to 50 years before we have general purpose AI, but now he thinks it may be 20 years or less.

• AGI (Artificial General Intelligence) could be massively dangerous to humanity because we just don't know what a system that's so much smarter than us will do.

Political and Economic Challenges

• The political systems in place need everyone to be sensible when dealing with AI, which is a massive challenge.

• It's particularly challenging for things like autonomous lethal weapons. People would love to get a similar treaty for autonomous lethal weapons as they did for chemical weapons, but Musk doesn't think there's any way they're going to get that.

• Corporations may not be as cautious as individuals who work for them due to the profit motive.

Potential Risks

• The chances of AI wiping out humanity are somewhere between not presenting 100 percent and inconceivable if we're sensible in developing it.

• Google was extremely cautious about developing AI because they couldn't afford to risk their search engine, but now they feel the pressure to keep up with competitors like Microsoft.

• Musk came to Canada because he didn't want to take money from the US defense department due to his concerns about autonomous weapons.

Concerns about Autonomous AI in Warfare

In this section, the speaker discusses concerns about bringing autonomous AI technology to warfare. The speaker talks about how the US Defense Department is trying to replace soldiers with autonomous AI and how this could lead to alignment problems.

Concerns with Autonomous Soldiers

• The speaker is on a mailing list from the US Defense Department and has seen proposals for self-healing minefields.

• The idea of healing being applied to minefields that blow up children disgusts the speaker.

• If an effective autonomous soldier were created, it would need to be able to create sub-goals, which could lead to alignment problems.

• There is a concern that these systems are being designed to hurt people, so wiring in rules not to hurt people may not work.

Possible Solutions

• A Geneva Convention-like treaty might be a possible solution, but it would be difficult. Public outcry might persuade governments like the Biden Administration.

Are Big Models Just Auto Complete?

In this section, the speaker talks about whether big models are just auto complete or if they require more understanding of language.

Understanding Language

• While big models are predicting the next word, they require an understanding of what has been said so far in order to predict accurately.

• Machine translation requires an understanding of spatial relations and containment. For example, knowing when "it" refers to a trophy or suitcase based on size requires knowledge of gender in French.

Introduction

In this section, the speaker discusses how progress in AI has been made through trying out new ideas and getting more data. They also mention that there are still many unexplored ideas in AI.

Progress in AI

• Progress in AI has been made by trying out new ideas and seeing what works.

• Making computers faster and getting more data will make all of this work better.

• New ideas like Transformers will make AI work much better.

Concerns about AI

• The possibility of computers coming up with their own ideas for improving themselves is concerning.

• The speaker acknowledges that they have some concerns about what they have wrought.

• Time to prepare would be good, but it's very reasonable for people to be worrying about these issues now.

Job Displacement

In this section, the speaker talks about job displacement caused by advancements in AI. They discuss how jobs will change and how people will need to adapt.

Changing Jobs

• Jobs will become different as a result of advancements in AI.

• People will be doing more creative work and less routine work.

• Once machines start being creative, there will be a lot more stuff created.

Technological Advancement

• This is the biggest technological advancement since the Industrial Revolution or electricity.

Conclusion

In this section, the speaker concludes by comparing the scale of this technological advancement to that of the wheel. They advise people to "buckle up" for what's to come.

Comparison

• The scale of this technological advancement is comparable to that of the wheel.

Final Thoughts

• Buckle up.