Stephen’s view
Here’s a question from Marta in Poland.
Hi Stephen,
After the controversial comments by the CEO of an AI company that you mentioned on your programme, a heated debate arose all over the world. In your view, is it just hype, or is there a grain of truth in the statement that the human species is doomed by the uncontrolled development of AI?
I truly can't wrap my head around it. On the one hand, it sounds scary. I feel as though I was forced to take part in a huge experiment I haven't consented to.
On the other hand, I can't help feeling that it's so... childish. I imagine a child spilling a bucket of water on the floor, jumping into it on purpose to see what happens, and then crying because they got wet from head to toe.
Why develop the thing and then spread panic about it?
Take care,
Marta
Perhaps the idea of Artificial Intelligence causing existential risk, or an apocalypse-type scenario is relatively new to most people, but I have been aware of this risk for many years. I remember watching this TED talk by the philosopher Sam Harris, which is ten years old, and thinking that it made sense. It still does. It’s impressive how this talk seems like it could have been made today, considering how much development there has been in AI over the last few years.
Ten years ago, people rarely spoke about AI, and certainly people didn’t use it. It has been less than four years since ChatGPT became publicly available, and other large-language models followed. In 2021, ChatGPT didn’t exist, and by 2026, 44% of Americans had used it. Including other AI models, the number would be higher.
It has been depressing for me to hear people dismiss talk about the existential risk of AI as ‘hype’. However, I can understand why people might think this, especially if they are hearing about it for the first time. It is an extremely strange situation for the bosses of the AI companies to be asking for their business to be regulated. Because of how strange this seems, many people have assumed that they are not doing it out of a genuine concern for minimising risk, but that they are looking to create ‘hype’ to make their companies seem even more important, more valuable, and to allow them to dominate the world even more.
I disagree that this is happening. I think that the reason that these CEOs are calling for regulation is because they genuinely see serious threats coming from the AI that they are creating.
It would require a much longer essay to write in detail all of the potential threats coming from AI, so I won’t do that here. However, I do believe that they are real, and I do think that there needs to be much heavier and much better regulation of the AI companies and AI models. This should also include AI companies being held responsible for the consequences of their AI.
One common analogy for the moment we are in is to the Manhattan Project. If you’ve seen the film “Oppenheimer”, you might remember this. The scientists building the first nuclear bombs thought that there was a very tiny possibility that a chain reaction could occur that would ignite the atmosphere and destroy the world. They thought that the probability was small enough to ignore. But did they have the right to make that decision for everyone else on Earth?
Today, many (possibly even most) of the scientists, researchers and engineers who are working on AI say that there is a chance that AI could cause an extinction-level event. Many of the top scientists even put this probability at 10% or 20%, although they also admit that it isn’t possible to calculate the actual risk, because this frontier has never been crossed before. Even the AI CEOs themselves agree that this is a danger (although I haven’t heard them say a percentage probability).
Do I think that AI is going to cause an apocalypse? No. But if the people who know the most about this technology say that it is a possibility, then we should absolutely take them seriously. Even if the risk is only 1%, then we should stop it.
As for your last question, Marta…
Why develop the thing and then spread panic about it?
I think that the reason that these AI companies are simultaneously creating AI and also talking about the dangers of it is because this is a unique situation. There’s never been anything like it. AI is essentially the natural continuation of the development of technology. It is natural for people, universities and companies to continue to develop technology, and AI is a logical step. Making the decision to stop development, or even to slow it down, is an unnatural decision to make and difficult to implement when there are thousands of organisations around the world developing it. Especially because the benefits of AI are so clear; it is surely true that AI will help to find cures for diseases and help to improve lives in various ways.
Luckily, there are relatively few companies (OpenAI, Anthropic, Microsoft, Google, etc.) who are creating the most advanced AI models that have the potential to create the most danger, so I believe that regulation is possible. Also, although the US and China don’t have much in common, one thing that they do share is that the leaders of both countries don’t want to die in an AI apocalypse. A shared sense of mortality might allow them to agree on regulations and safeguards, in a similar way to agreements on nuclear weapons.
I really hope that I am wrong, and that AI is not as risky as it seems. But if it were up to me, we would pause AI development completely until we understood the risks better.
Have a great weekend.
Stephen
P.S. Sorry if I have been slow to respond to your email recently. I'm currently in the process of moving to Spain for a year. I'll be driving all the way through France tomorrow and Sunday!
