AI detectives are cracking open the black box of deep learning

Introduction to Neural Networks

In this section, the speaker introduces neural networks and how they work.

What are Neural Networks?

• Neural networks are a type of machine learning that use a network of neurons to process data.

• They are particularly good at image recognition and have applications in autonomous cars and genetic sequencing.

• The network is made up of interconnected neurons that mimic the brain's decision-making process.

• Each neuron has a threshold called a weight where it makes a decision based on input data.

How Do Neural Networks Learn?

• Once the network is trained, it can recognize patterns in new data by firing when it sees something it recognizes.

• Back propagation is used to improve the network's accuracy. It involves sending information back through the network when it makes an incorrect prediction.

• Breakthroughs in neural networks came when researchers stopped trying to make them biologically accurate and focused on processing power and examples.

Understanding Neural Network Decision-Making

In this section, the speaker discusses how neural networks make decisions and why they can be difficult to understand.

Why Are Neural Networks Difficult to Understand?

• While neural networks can be accurate, they can be difficult to understand because they engage in complex decision-making processes.

• Many researchers think of neural net decision making as a terrain of valleys and peaks with each piece of data represented by a ball.

• This complexity means that neural nets are essentially black boxes - we don't really know how they think or why they make certain decisions.

Solving the Black Box Problem

• Researchers are working on ways to solve the black box problem, such as creating toolkits that allow them to examine individual neurons in a neural network.

• By examining the weights that make a neuron fire, researchers can gain insight into how the network is making decisions.

• Some neurons learn complex abstract ideas like face detection, which is not something you would expect an individual neuron to be able to do.

Understanding AI Decision Making

In this section, the speaker discusses how to understand what an AI is thinking and how one professor trained an AI to play a video game using human insights.

Training an AI to Play Frogger

• One professor trained an AI to play the video game Frogger.

• The AI played the game extremely well, but it was hard to know what it was deciding to do.

• It's difficult to understand its sequence of decisions in a dynamic environment.

Using Human Insights for Better Results

• Instead of trying to get the AI itself to explain itself, people were asked to play the video game and say what they were doing as they played it.

• The state of the frog was recorded at the same time.

• Neural networks were used to translate those two languages - the code of the game and what they were saying.

• This information was then imported back into the game playing network, providing it with human insights.

Importance of Trust in Research

• Trust is important when working with neural networks because if you have a result and don't understand why it made that decision, research cannot advance.

• It's essential to ensure there are no spurious details that could throw things off.

Pushing Science Forward

• While we may not achieve a global understanding of what a neural network is thinking anytime soon, if we can gain a sliver of this understanding, science can push forward and these neural networks can play.