Ville V. Kokko
Lately, we’ve been hearing (again) about how AI is going to go rogue and probably kill us all. This has long been a favourite prediction of the “AI safety” community, movements like the rationalists, and (weirdly) the AI companies themselves, but it seems to be becoming ever more mainstream now after incidents such as OpenAI agents breaking into the Hugging Face servers and Jacob Coxon quitting Anthropic and publicly talking about how they believe they are toying with human extinction.
What should we think and do about all this – especially if we’re more concerned about concrete rather than speculative problems with AI?
A lot of people fall into either of two camps: those who strongly believe in artificial superintelligence and are just waiting for it to happen, and those who look at the technology as it is now and refuse to believe in such science fiction-sounding scenarios. Mind you, not everybody thinks they know. Even if you think a leap to world-ending superintelligence is unlikely, you could still say even a small chance of it needs to be taken very seriously.
(If you want to know more about different takes on the issue, there was a good video published a while ago featuring a debate between people with very different views. It’s still framed around AI doomsday scenarios, but at least it also features people disagreeing with them and wanting to draw attention to other concerns.)
You might think that people worried about the effects of AI for different reasons would have common ground in hoping to, say, slow its development. While they can agree about some things, surprisingly often, they are at odds. You could say one side tends to view superintelligence as a threat, whereas the other views the (fictional) idea of superintelligence as a threat.
Should we focus our attention and resources on preventing a future catastrophe that is so vast that it overrides everything else in importance – or is that only a distraction from all the harm AI is causing now, an excuse for AI companies to do whatever they want in practice when everyone is distracted by this future vision that seems inevitable? Is it important for us to understand the theory of how dangerous artificial superintelligence could be if we got to it – or should we be more concerned that even the concept of artificial “intelligence” is harmfully misleading, to say nothing of this nebulous concept of artificial superintelligence?
Advances in current AI systems can be seen as supporting the possibility of artificial superintelligence, but that depends on the lens through which they are viewed. On the one hand, these stories match the narrative that AI keeps getting better in ways that the believers in superintelligence have predicted. On the other hand, even now, everything that’s happened also fits a much narrower paradigm where AI is getting better at some limited tasks but can’t cross the border into independent, general, human-like action and intelligence.
I’m not here to tell you what will happen in the future, though certainly FRACTAI is focused on the existing challenges and possibilities of AI rather than the speculative scenarios. I can say something about how to deal with the existence of the different views, however.
The publicity around the recent events should not be made all about apocalypse scenarios. The discussion is an opportunity to bring up other issues with the way AI companies are being run. However, it also shouldn’t be seen as a zero-sum contest between the different concerns.
A few people would be willing to bet everything on the possibility of hypothetical benefits of future super AI. Everyone else should be able to agree that AI being developed so recklessly that it accidentally hacks other companies is a sign that something is going wrong – on top of every other sign that we have already seen.
However that is done – unfortunately, I can’t tell you what all the solutions should be in this blog post – we need to rein in the way we are blindly being rushed forward with AI development and deployment. Even if artificial superintelligence is only a mirage we will never reach, the idea is already doing harm.