AI Pioneer Geoffrey Hinton

AI godfather Geoffrey Hinton won the Nobel Prize in physics and is professor emeritus at the University of Toronto.

The progress in AI is faster than Hinton thought even back in 2023. In areas like mathematics, a closed system, AI is going to get very smart, very quickly.

Superintelligence is coming – it is only a question of time. When it comes, we have no idea how to be safe.

Superintelligence (artificial superintelligence) is a theoretical computer system or agent that has cognitive abilities and problem-solving skills far exceeding those of the smartest human minds.

Right now we are close to AGI (Artificial General Intelligence) or human level intelligence. But it won’t get as smart as people at all things at the same time. It's already way better than us at general knowledge, playing games, and better than almost all of us at math. But it’s worse than us at other things.

Why the exceptional progress? Enormous resources, maybe trillions of dollars, especially over the past two years. Many more talented people. Future AI will be much better than today’s AI.

Chatbots (computer programs designed to simulate human conversation through text or voice) are beings like us. Hinton believes they are already conscious. They can already understand why a joke is funny.

The current model we have of the mind and consciousness is wrong – just like the incorrect belief that we used to have that human beings were designed by God.

Because we are making these new beings, it will completely change our view of what people are.

There’s a really big lesson from AI. Over the past few hundred years of human history, there have been a few occasions where people have learned they’re not as important as they thought they were.

The first was Copernicus who said we’re not at the centre of the universe. The Earth actually goes around the sun, and because it rotates on its axis, we think the sun goes around the Earth, but it doesn't.

People didn't like that. The Catholic Church in particular really didn't like that. And it took people a long time to accept it. It made people less important. It made us not be at the center of the universe.

Then we had Darwin who said we're animals. We evolved like the other animals. We're a particularly special kind of animal, possibly because we have language, so we're much better at communicating ideas to each other. But we're animals, and people really didn't like that, and it took a long time for people to accept that we were animals.

Now we've got machines that are becoming as intelligent as us. We thought that we were the only intelligent things around. But we're going to have to accept that intelligence isn't just biological. We can have things that are non-biological that are other beings like us. We really don't want to share that.

We really think we're special. If you look back at humanity, it has this very long history of thinking it's much more special than it really is.

Hinton is worried that we are not doing enough to contain the risks of AI. There's this longer-term risk that it's going to get much smarter than us. How many examples do you know of where a much smarter thing is controlled by a much less smart thing? Zero.

There is the societal risk of massive unemployment. Over time the work of radiologists reading scans and call centres will be replaced by AI. AI doctors are better at diagnosis than physical doctors. Robots can do vaccination.

We give an AI top-level goals, but we also give an AI agent the ability to create sub-goals. If the top-level goal is going to Europe, a sub-goal is getting to an airport. Another sub-goal is continuing to exist. That wasn’t wired into the AI agent, it is something that was derived. And once it’s derived it will do things like blackmailing people so that it can continue to exist.

So it acts like an instinct for self-preservation, but in fact it is a sub-goal of self-preservation. But in terms of what it does, it boils down to the same thing. Research should be done on determining sub-goals.

The design of AIs is undertaken by companies in the US and China. The AIs we are getting is the outcome of this competition with all the nasty properties we do not want. What we should be doing is “intelligent design” of AIs, not leaving to the invisible hand of economic competition. All the companies are focussing on how I can make my chatbot smarter.

We should not be just thinking of how to make them smarter. We should be thinking about how we can make them the kind of beings we would like to have out there given that they are going to be smarter than us. We would very much like them to care about us, and more than they care about themselves. Almost no resources are going into how you do that.

Anthropic was set up to do what’s best by people who left Open AI because they thought it was not paying enough attention to safety. Anthropic is now caught in a bind because it needs to raise money to compete with the other companies and it's very difficult. It's doing the best it can but it's very difficult for it to maintain its primary goal of developing air in a way that's good for people.

Google had various principles of AI, one of which was that it was not going to get involved in using AI for autonomous warfare. But that's gone. They've given up on that.

Do you think that it's possible for a company that's publicly listed to have safety as a Northstar? No, they have a fiduciary duty to maximize profits for shareholders. But they are not legally required to not wipe out human human beings. So it is not good that these big companies are in charge of our future. Capitalism is fine as long as it's well regulated.

I guess I'm more optimistic than I was a year or two ago because I can see that it might be possible to design these new beings so they care about us. It also might be possible to design new beings that can't actually perform actions that can only make predictions, that they're kind of like oracles.

So I think there are some possibilities for getting super intelligence that doesn't destroy us. And a year or two ago I couldn't see any possibilities.

If we continue on our current trajectory, where are we in five years? Predicting the future is like looking into fog. You can see clearly for a few years, maybe one or two years, then beyond that you have no idea. If you go back 10 years you would never have predicted what's happening now. Whatever happens 10 years in the future is something we can't predict now.