Technology
PUBLISHED
1
There has been a lot of chat lately around the topic of super-intelligent artificial intelligence (AI), and the risks it may pose to humanity. It's difficult to scroll through the internet, past endless AI-generated content, without seeing warnings from insiders that AI will destroy humanity
The latest round of hype is overblown, with genuinely super-intelligent AI deemed unlikely to emerge from the current round of large language models (LLMs). Nevertheless, superintelligence is a subject that has been thoroughly explored by academics over the years, and they have plenty of warnings in store for those seeking to make what could easily turn out to be our destroyer.
One paper on the topic comes from philosopher Nick Bostrom, whose other notable work includes the influential paper on the simulation argument. Bostrom's work has also focused on risks to humanity at the Future of Humanity Institute, and one of the risks he has turned his attention to is those posed by artificial intelligence.
While many a paper and science fiction work have focused on the risk of malevolent AIs with their own nefarious goals, in one paper Bostrom focused a little more on the dangers posed by AIs pursuing a seemingly benevolent goal, and chosen by a human: manufacturing paper clips.
Sure, it feels right that humanity should be killed by the pursuit of office equipment, but how?
Now a lot of you are nodding, saying "yes, it makes sense that stationery would be our downfall", but let's go through the argument anyways.
To start with, we are not talking about AI chatbots, but an advanced intelligence vastly superior to our own.
"A superintelligence is any intellect that is vastly outperforms the best human brains in practically every field, including scientific creativity, general wisdom, and social skills," Bostrom writes in his paper.
"This definition leaves open how the superintelligence is implemented – it could be in a digital computer, an ensemble of networked computers, cultured cortical tissue, or something else."
With these superintelligences, he argues, you would need to be incredibly specific with the goals and motivations you set. If you don't, you may end up in a situation that quickly gets out of the control of your puny, human intelligence.
Bostrom chose paperclips as an example because the goal of creating them seems pretty innocuous, but could nevertheless could go dramatically wrong when the AI given that goal is far smarter than the human who set it.
With the overall goal of making paperclips, the AI may conduct this task by "transforming first all of Earth and then increasing portions of space into paperclip manufacturing facilities".
"The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off," Bostrom explained to HuffPost in 2014. "Because if humans do so, there would be fewer paperclips."
"Also, human bodies contain a lot of atoms that could be made into paperclips. The future that the AI would be trying to gear towards would be one in which there were a lot of paperclips but no humans."
The example is of course used to show how even seemingly-innocuous goals could go wrong, and the care humanity must take in the initial phase of creating a super-intelligence, before it's too late.
"More subtly, it could result in a superintelligence realizing a state of affairs that we might now judge as desirable but which in fact turns out to be a false utopia, in which things essential to human flourishing have been irreversibly lost," he added.
"We need to be careful about what we wish for from a superintelligence, because we might get it."
It may be an overblown example, but if the creators of current AIs truly do believe they are on the cusp of creating something more advanced, then maybe it's worth a little look over. We want paperclips, sure, but not so many that it requires humanity to be enslaved to a paperclip-manufacturing overlord in order to get them, after all.
