AI apocalypse or overblown hype?

The rapid speed at which artificial intelligence (AI) continues to develop highlights the importance of understanding, and acting on, the potential risks it poses

When generative artificial intelligence (AI) burst onto the scene, social media, traditional media and water cooler conversations were awash with predictions about AI’s implications, including the risks and dangers of large language models like ChatGPT.

Since then, advances in large language models have been significant and rapid and their increasing power —their ability to perform aspects of scientific research and to displace the need for humans to write code— has surpassed even expert expectations.

Headlines alternate between hype and hysteria. Countries likewise are taking drastically different steps to either tame AI or herald in the new world it could bring.

AI pioneer, Geoffrey Hinton, who resigned from Google in 2023 so he could speak freely about the technology’s dangers, continues to warn of the risks of artificial intelligence.

“AI can cause us three kinds of harm. One is bad actors using it to do bad things like cybercrime, corrupting elections, or launching nasty autonomous weapons,” says Hinton.

“Another is causing massive loss of jobs—the large companies aren’t thinking about what happens when they replace most workers with AI. The third thing is AI itself taking over because it’s a better form of intelligence.”

Within three decades, Hinton believes there is a 10% to 20% chance that AI will lead to human extinction.

Hinton’s remarks echo those of another ‘Godfather of AI’, Yoshua Bengio, who, in March 2023, called for a pause on the development of AI models. Elon Musk also supported a six-month pause in AI development—but that never happened.

The AI arms race

Instead, AI development has forged ahead and, with China now outpacing the US in AI research, many in the national security world are concerned about an AI arms race.

America is worried about China and other countries matching its AI capability and, as a result, limited the sale of certain computer chips used to create the models. But that was only a temporary setback. China quickly worked out other ways to use the chips it still had access to.

In this climate, America shifted. When the administration changed, America moved from a position that appeared to favour a balance between regulation and innovation to more strongly prioritizing American AI-supremacy.

Many individuals and Big Tech companies contend that AI is more likely to be a force for good and we should “let it rip”. And, there’s no doubt that AI has brought substantial gains. AI models can now accomplish tasks in minutes that would take a skilled professional a full day to complete.

But have we reached an AI turning point? Is it the beginning of the end for humanity? Should we be worried about AI, and if so, why?

We, along with the world’s experts, are uncertain and divided.

There are two camps of thinkers with different takes on AI’s immediate and longer-term risks – the ‘AI Safety camp which is concerned about large scale catastrophic risks, and the ‘AI Ethics’ camp which focuses on more immediate implications such as social justice.

Alignment and ethics

Those in the AI Safety camp generally believe ‘mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.’

Anthropic, one of the few major AI companies, is representative of the camp most worried about catastrophe. When they attempted to restrain the US Department of War from using its technology for domestic mass surveillance the government designated it a ‘supply chain risk’–indicating the challenging politics of both needing sales to sustain the firm, but being deeply concerned about the impacts of its use.

Others in the AI safety camp—like Musk, Hinton and Sam Altman (the CEO of OpenAI)—fear AI could be turned into a weapon of mass destruction by autocratic nations, or that a self-aware AI could be destructive. But when Anthropic was designated a supply chain risk, Altman swooped in, allowing the Department of War to use OpenAI for military applications.

Swedish philosopher, Professor Nick Bostrom, believes that in the not too-distant future, AI may outstrip human intelligence, creating a dangerous superintelligence. These risks are only exacerbated by giving such tools access to high-risk military and surveillance systems.

In contrast to the views common in industry, there is a contrasting camp of academics and others who believe the risks of an AI apocalypse pale in comparison to the harms AI may be causing now.

AI superintelligence and slop

Some argue that the word “AI” is typically used to sweep the limitations of the tools under the rug, even going so far as to suggest that there are relatively few real uses for large-language models. They point not only to the limitations of today’s tools, but to the expense and human capital spent on building them.

Other issues critics raise include concerns that humans will deskill by overly relying on tools, and that when the tools don’t produce good results, the people will have adjusted to uncritically accepting the results. In Victoria, a King’s Counsel, his junior counsel and their instructing solicitor were referred to legal professional bodies after AI ‘hallucinations’ were detected in their court submissions.

Likewise, writing that is stylistically coherent but substantively vacuous is now widespread. Consulting companies and academics have used AI to produce submissions to Government that are inaccurate and show oversights that a human likely would not have made.

AI’s history reveals sharp jumps in technological capabilities that apparently herald unlimited growth. Historically, these jumps have been bookended by long AI winters where progress slows considerably. On this issue, experts are split into two camps. Some, like Gary Marcus, say large language models are reaching their limits, but Sam Altman believes we are on a trajectory to super AI.

For better and for worse

We have reached a turning point in AI’s ability to impact the world. For better and for worse. We believe that both camps raise issues worth pondering, whether the future of AI is a further large increase in capabilities or not.

It’s time to take seriously the full suite of risks, and listen carefully to the opposing viewpoints—the future is yet uncertain.

Dr Simon Coghlan: Senior Lecturer in Digital Ethics, Centre for AI and Digital Ethics and School of Computing & Information Systems, Faculty of Engineering and Information Technology, The University of Melbourne.

Dr Shaanan Cohney: Lecturer, Cyber Security, Faculty of Engineering and Information Technology, The University of Melbourne; Center for IT Policy, Princeton University.

Dr Shaanan Cohney: Senior Lecturer, Cyber Security, Faculty of Engineering and Information Technology, University of Melbourne; Center for IT Policy, Princeton University.