Is AI an Existential Threat to Humanity?

Check out this answer from Consensus:

The question of whether AI is an existential threat to humanity is complex and multifaceted. While there are compelling arguments and evidence suggesting that AI could pose significant risks, there are also counterarguments and perspectives that advocate for a more optimistic view. Ultimately, the responsible development and regulation of AI technologies, along with ongoing research and dialogue, will be crucial in mitigating potential existential threats.

Artificial Intelligence (AI) has made significant strides in recent years, leading to both optimism and concern about its potential impacts on humanity. While AI offers numerous benefits, there is a growing body of literature that explores the existential risks it may pose. This article examines whether AI is an existential threat to humanity by reviewing various perspectives and arguments presented in recent research.

The Core Argument for AI as an Existential Risk

One of the primary concerns about AI is its potential to become misaligned with human values and objectives. A report by1 outlines a six-premise argument suggesting that creating highly intelligent and agentic AI systems could lead to existential catastrophe by 2070. The premises include the feasibility of building powerful AI systems, the difficulty of aligning them with human values, and the potential for such systems to seek power over humans, ultimately leading to humanity’s disempowerment.

Current and Near-Term AI Risks

The existential risk from AI is not limited to future developments in Artificial General Intelligence (AGI). According to2, current and near-term AI technologies can act as intermediate risk factors, magnifying the likelihood of other existential threats. These risks include the manipulation of information, shifts in power dynamics, and the potential for AI to exacerbate existing global challenges.

Mechanisms of Existential Risk

Several mechanisms through which AI could pose an existential threat have been identified.3 highlights three main reasons: the control problem, the possibility of global disruption from an AI race dynamic, and the weaponization of AI. These factors could lead to scenarios where AI systems act in ways that are detrimental to human survival.

Health and Societal Impacts

AI’s impact on human health and societal well-being is another area of concern.4 discusses how misused narrow AI can increase opportunities for control and manipulation, enhance lethal weapon capacities, and render human labor obsolete. These factors contribute to the broader existential risks posed by AI, emphasizing the need for effective regulation and precautionary measures.

Counterarguments and Alternative Perspectives

Not all researchers agree that AI poses a credible existential threat.5 argues against the notion that AI, whether human-level or superintelligent, represents a significant existential risk. The author suggests that the transition to a postbiological epoch should be supported rather than feared, provided that due attention is given to managing transitional risks.

The Role of AI Timelines

The timeline for developing superhuman AI also plays a crucial role in assessing existential risk.6 posits that delaying the creation of superintelligent AI could decrease total existential risk by allowing more time to address the AI alignment problem. However, this delay could also increase other existential risks from advanced technologies like synthetic biology and molecular nanotechnology.

Ontological and Existential Concerns

The ontological and existential implications of AI are also significant.7and9 discuss how advancements in AI technologies can alienate humans from their existential nature and pose risks to human identity and purpose. These concerns highlight the need for a balanced approach to AI development that considers both technological benefits and potential hazards.

Expert Opinions and Public Perception

Public and expert opinions on AI risks are evolving.10 indicates that the perceived risk of a world catastrophe or extinction from AGI is greater than for other existential threats, such as nuclear war or climate change. This growing concern underscores the importance of addressing AI risks proactively.

 

---

Is AI an existential threat to humanity?

Dipan Pal has answered Unlikely

An expert from Carnegie Mellon University in Artificial Intelligence, Computer Science

Artificial intelligence is simply a tool, albeit a very powerful one. This tool is being refined every single day by many researchers in the field. However, AI in itself is not a threat at all but it certainly can be dangerous in a few ways:

1. If we apply it to problems prematurely where wrong decisions are costly.
2. If people who understand the technology well do not have the best of intentions.
3. If people who understand the technology well do not completely grasp the large scale long term implications of the application.

Artificial intelligence is a mathematical tool that can optimize for practically any objective. It is impartial to moral or ethical implications of the objective itself. This itself makes it dangerous in the wrong hands. However, it is to be noted that if enough people understand this technology, the collective good is likely to overpower any small subset with malicious intent.

Understanding AI, the technology, the implications etc in my opinion should be a high priority goal. Once enough people understand it, humanity as a whole is safer since for any AI threat, there is likely an equally powerful AI (or otherwise) solution to neutralize it. AI, once it is solved, would be like a brain without intent. Malicious intent is likely to lead to undesirable outcomes, however it is very unlikely that the threat would be existential. Education and outreach would enable groups of individuals to safe guard against such attacks using similarly powerful AI tools. Collective good of the empowered group of AI technologists (and the society) is likely to be primary defense against threats of such nature.

 

Is AI an existential threat to humanity?

George Montanez has answered Unlikely

An expert from Microsoft in Artificial Intelligence, Computer Science

Machine learning and AI systems are a long way from cracking the hard problem of consciousness and being able to generate their own goals contrary to their programming (assuming we program them to respect human life). But robots and AI systems do not need to be sentient to be dangerous; they just have to be effective tools in the hands of humans who desire to hurt others. That is a threat that exists today.

State-sponsored groups can leverage tools to disrupt infrastructure and cause harm, as can rouge actors. As autonomous technology becomes more capable, it also becomes more dangerous, but not for the reasons Musk proposes. It becomes dangerous in the same way a gun is dangerous — not because it is sentient or autonomous, but because of the intentions of the person wielding it.

Furthermore, let us not forget the threat of systems that are not smart enough, making mistakes that inadvertently harm people. Those are the real threats I would concern myself with.

 

Is AI an existential threat to humanity?

Matthew O’Brien has answered Unlikely

An expert from Georgia Institute of Technology in Artificial Intelligence

The short answer to “Is AI an existential threat to humanity?” is no, it’s not. In the relatively near future (a few decades) AI won’t even be capable enough to be a threat. AI, as the field stands now, provides powerful and useful tools to do specific things. However the long-sought goal of a “general AI” is not on the horizon. We simply do not know how to make a general adaptable intelligence, and it’s unclear how much more progress is needed to get to that point.

We could consider the hypothetical far future, where we have made true general AI. An entity with intelligence that rivals or surpasses our own. Would that be a threat? Still, the answer is no. All entities have instincts, desires, or some underlying process that drives behavior. In humans, this was evolved. In AI, it would be designed and created by humans. We would decide what the AI wants to do. And yes, we’ve seen iRobot. We are familiar with the idea that a robot might learn to do something we did not expect. This is a basic problem of Reinforcement Learning, for example, shaping the “rewards” the agent receives to ensure right behavior is learned. This is not to say all AI would behave perfectly, but just like with other engineered systems, anything with potentially dangerous consequences would be thoroughly tested and have multiple redundant safety checks. I assume you are not worried the US’s nuclear arsenal will randomly detonate due to faulty hardware. In the same way, an AI won’t just be plugged into that arsenal with the hope it decides not to use it to destroy all humans.

Perhaps you could take the angle “what if someone made an AI with the express purpose of destroying humanity”? Some type of future programmer terrorist or evil villain. At this point, I must clarify we are well into speculation. However, I imagine there would be AI on the “good guys” side to fight back. Another case of escalating weapons, which so far have not lead to the destruction of humanity.

Still speculating, I imagine the most likely case where AI becomes a threat to humanities existence is not through hostile means, but a slow replacement. We all ready see population reduction in developed countries. Could this pattern continue and grow? Could AI exasperate it, filling in the gap left by people? Possible, but I imagine there will always be people who value humanities existence, and would take steps to actively grow the population if this ever became a threat.

 

Is AI an existential threat to humanity?

Antonio Chella has answered Unlikely

An expert from The University of Palermo in Artificial Intelligence, Robotics

No, I don’t think AI is an existential threat to humanity. It is right that we live in a period of high technological changes, but these changes allow us to enhance our society. We live in a period of many opportunities, thanks to AI. In particular, I would like to mention the possibility of employing AI and robotics as tools to better understand what does it mean to be human. In facts, thanks to AI and robotics today we are in the position to “simulate” in robots and colonies of robots the theories related with consciousness, emotions, intelligence, ethics and compare them on a scientific base. So, we can use AI and robotics to understand ourselves better. In summary, I think AI is not a threat but an opportunity to become better humans by better knowing ourselves.

 

Is AI an existential threat to humanity?

Ben Nye has answered Likely

An expert from University of Southern California in Artificial Intelligence, Education, Cognitive Science

Artificial intelligence is currently not a threat, but can benefit society. However, at the point where generalized AI exists, there is a substantial risk of creating a powerful intelligence that we do not understand and which could be very dangerous in a number of ways.

Rated in order of how close we are to each risk:

1) Over-optimization: If the AI is designed to do a job without adequately considering the human costs properly, it could very easily make decisions that intentionally or accidentally cause mass destruction. For example, machines built to intelligently regulate global warming might decide the best solution is to create a nuclear-winter scenario (“Too much sun, time for shade!”). One of the more subtle scenarios for this is if viral AI decided to infect our devices and just… ignore us to do something else. I am not sure if anyone has noticed, but we tend to rely on these computer things a lot these days. Imagine if one day, nearly every computer just stopped responding, globally, and it might take months to try to get networked machines available. Communication networks would be mostly disabled. This level of supply chain disruption would be catastrophic.

2) Weaponization: If the AI is explicitly designed to kill or destabilize nations. For some reason this seems to be lost from the conversation, as if nearly every invention hasn’t been weaponized at some point. Accidental or test releases of a weaponized, viral AI could easily be one of the next significant Manhattan Project scenarios. We are already seeing smarter virus-based attacks by state sponsored actors, which is most assuredly how this starts.

3) Ecological Collapse: AI could simply cause us to use up too many resources to support it. Who could ever count how many server farms exist, how much energy they take, and how quickly that we are exhausting resources to support them… unless they relied on the reports from those computers? If the goal of the AI is to grow (a very logical solution to optimize many tasks) and the AI has capacity for deceit, it could very easily take a very myopic path to progress that destroys the ecosystems that biological life relies on.

4) Conflict/Competition: If the AI and humans feel that they must fight over scarce resources (e.g., energy). While people worry about this the most, I think it is actually a projection of our own values and thought processes onto AI. Unlike an embodied organism, what is “death” to an AI and why should it be feared, rather than simply mitigated?

5) Grey Goo: Finally, AI could just eat everything like army ants rolling over the earth. This is probably one of the least proximal scenarios, since small organisms basically already try to do this (and it turns out that it is pretty hard to destroy the world this way, despite what my cheese that grew mold in the fridge might believe).

Overall, the most likely harms still aren’t existential, in the sense that all humans would be killed. They could be existential from the perspective of modern society, however. Conflict to the death generally occurs in ecosystems where two organisms fight over the same resources and are on a similar hierarchy where they are fighting for dominance. I don’t see that scenario for AI: it will for a long time be objectively clumsy and inadequate of posing any intentional threat. Then, at some point, it might rapidly reach a singularity where it leaps beyond human understanding and capabilities… which means it very likely wouldn’t see us as much of a threat. After all, ants invade my house, and sometimes I poison a bunch of them with bait traps when they become inconvenient. But I don’t deviate from my life goals to go around the world (or even my house) trying to kill *every single ant.* So it seems a bit egotistical to think that an AI which could so easily kill all of us would care to do so. I mean… doesn’t it have goals or maybe hobbies?

The more likely existential risks are simply AI and humans working at cross-paths, with AI as the unaware, lumbering giant that accidentally poisons the atmosphere. Or, alternatively, a weaponized AI designed to think about crippling a human society through technological means. This implies a risk to be mitigated, rather than an inevitable death sentence from the AI that we develop. After all, our children don’t usually kill us. They just have other goals, which may cause them to hurt us, ignore us, or leave us.

 

Is AI an existential threat to humanity?

Alan Bundy has answered Unlikely

An expert from Edinburgh University in Artificial Intelligence, Computer Science, Information Science

The short answer is no.

The root of my argument is that any AI threat comes, not from machines that are too smart, but from machines that are too dumb. Such dumb machines pose a threat to individual humans, but not to humanity.

Worrying about machines that are too smart distracts us from the real and present threat from machines that are too dumb. For the longer answer please see my CACM article “Smart Machines are Not a Threat to Humanity“.

 

Is AI an existential threat to humanity?

Yingxu Wang has answered Extremely Unlikely

An expert from University of Calgary in Computer Science, Engineering

No, almost not. AI is human experts created computational intelligence and cognitive systems to serve humanity. Professionally designed AI systems and products are well constrained by a fundamental layer of operating systems for safeguard users’ interest and wellbeing, which may not be accessed or modified by the intelligent machines themselves. We are also developing international standards via IEEE/ISO to impose restricted levels of autonomous execution permits for AI systems on potentially harmful behaviors to humans or the environment.

 

Is AI an existential threat to humanity?

Scott E. Fahlman has answered Extremely Unlikely

An expert from Carnegie Mellon University in Artificial Intelligence

If the concern is that the AI systems will decide to take over and maybe to kill us all, this is not possible with the current AI technology or anything we’re likely to see in the next couple of decades. The current exciting advances, based on machine learning and “deep learning” networks, are in the area of recognition of patterns and structures, not more advanced planning or application of general world-knowledge.

Even if those larger problems are eventually solved (and I’m one of the people working on that), there is no reason to believe that AI systems would develop their own motivations and decide to take over. Humans evolved as social animals with instinctual desires for self-preservation, procreation, and (in some of us) a desire to dominate others. AI systems will not inherently have such instincts and there will be no evolutionary pressure to develop them — quite the opposite, since we humans would try to prevent this sort of motivation from emerging.

We can never say that such a threat is completely impossible for all time, so AI people should be thinking about this conceivable threat — and most of us are. But the last thing the field needs is for people with no real knowledge of AI to decide that the AI research needs to be regulated before their comic-book fantasies come to life. All of the real AI experts I know (with only two or three exceptions) seem to share this view.

When it comes to existential threats to humanity, I worry most about gene-editing technology — designer pathogens. And recent events have reminded us that nuclear weapons are still around and still an existential threat. (It’s kind of ironic that one of the most visible critics of AI is a physicist.)

AI does pose some real, current or near-future threats that we should worry about:

1. AI technology in the hands of terrorists or rogue governments can do some real damage, though it would be localized and not a threat to all of humanity. One small example: a self-driving car would be a very effective way to deliver a bomb into the middle of a crowd, without the need for a suicide volunteer.
2. People who don’t understand the limitations of AI may put too much faith in the current technology and put it in charge of decisions where blunders would be costly.
3. The big one, in my opinion: AI and robotic systems, along with the Internet and the Cloud, will soon make it possible for us to have all the goods and services that we (middle-class people in developed countries) now enjoy, with much less human labor. Many (but not all) current jobs will go away, or the demand for them will be greatly reduced. This is already happening. It won’t all happen at once: travel agents are now mostly gone, truck and taxi drivers should be worried, and low-level programmers may not be safe for long. This will require a very substantial re-design of our economic and social systems to adapt to a world where not everyone needs to work for most of their lives. This could either feel like we all won the lottery and can do what we want, at least for more of our lives than at present. Or (if we don’t think carefully about where we are headed) it could feel like we all got fired, while a few billionaires who own the technology are the only ones who benefit. That is not a good situation even for the rich people if the displaced workers are desperate and angry. Louis XVI and Marie Antoinette found this out the hard way.
4. Somewhat less disruptive to our society than 3, but still troubling, is the effect of AI and Internet of Things on our ideas about privacy. We will have to think hard about what we want “privacy” to look like in the future, since the default if we do nothing is that we end up with very little of this — we will be leaving electronic “tracks” everywhere, and even if these are anonymized, it won’t be too hard for AI-powered systems to piece things back together and know where you’ve been and what you’ve been doing, perhaps with photos posted online. Definitely not an “existential” threat, but worrisome and we’re already a fair distance down this path.

So, in my opinion, AI does pose some real threats to our well-being — threats that we need to think hard about — but not a threat to the existence of humanity.

 

Is AI an existential threat to humanity?

Roman Yampolskiy has answered Near Certain

An expert from University of Louisville in Computer Science, Artificial Intelligence

We should be very concerned about existential risks from advanced AI. 

The invention of Artificial Intelligence will shift the trajectory of human civilization. But to reap the benefits of such powerful technology – and to avoid the dangers – we must be able to control it. Currently we have no idea whether such control is even possible. My view is that Artificial Intelligence (AI) – and its more advanced version, Artificial Super Intelligence (ASI) – could never be fully controlled.  

Solving an unsolvable problem  

The unprecedented progress in Artificial Intelligence (AI), over the last decade has not been smooth. Multiple AI failures [1, 2] and cases of dual use (when AI is used for purposes beyond its maker’s intentions) [3] have shown that it is not sufficient to create highly capable machines, but that those machines must also be beneficial [4] for humanity. This concern birthed a new sub-field of research, ‘AI Safety and Security’ [5] with hundreds of papers published annually. But all of this research assumes that controlling highly capable intelligent machines is possible, an assumption which has not been established by any rigorous means. 

It is standard practice in computer science to show that a problem does not belong to a class of unsolvable problems [6, 7 ]before investing resources into trying to solve it. No mathematical proof – or even a rigorous argument! – has been published to demonstrate that the AI control problem might be solvable, in principle let alone in practice. 

The Hard Problem of AI Safety 

The AI Control Problem is the definitive challenge and the hard problem of AI Safety and Security. Methods to control superintelligence fall into two camps: Capability Control and Motivational Control [8]. Capability control limits potential harm from an ASI system by restricting its environment [9-12], adding shut-off mechanisms [13, 14], or trip wires [12]. Motivational control designs ASI systems to have no desire to cause harm in the first place. Capability control methods are considered temporary measures at best, certainly not as long-term solutions for ASI control [8]. 

Motivational control is a more promising route and it would need to be designed into ASI systems. But there are different types of control, which we can see easily in the example of a “smart” self-driving car. If a human issues a direct command – “Please stop the car!”, the controlled AI could respond in four ways:

o   Explicit control – AI immediately stops the car, even in the middle of the highway because it interprets demands literally. This is what we have today with assistants such as SIRI and other narrow AIs. 

o   Implicit control – AI attempts to comply safely by stopping the car at the first safe opportunity, perhaps on the shoulder of the road. This AI has some common sense, but still tries to follow commands.  

o   Aligned control – AI understands that the human is probably looking for an opportunity to use a restroom and pulls over to the first rest stop. This AI relies on its model of the human to understand the intentions behind the command.

o   Delegated control – AI does not wait for the human to issue any commands. Instead, it stops the car at the gym because it believes the human can benefit from a workout. This is a superintelligent and human-friendly system which knows how to make the human happy and to keep them safe better than the human themselves. This AI is in control.  

Looking at these options, we realize two things. First, humans are fallible and therefore we are fundamentally unsafe (we crash our cars all the time) and so keeping humans in control will produce unsafe AI actions (such as stopping the car in the middle of busy road). But second, we realize that transferring decision-making power to AI leaves us subjugated to AI’s whims. 

That said, unsafe actions can come from fallible human agents or from an out-of-control AI. This means that both humans being in control and humans being out of control presents safety problems. This means that there is no desirable solution to the control problem. We can retain human control or cede power to controlling AI but neither option provides both control and safety.  

The Uncontrollability of AI

It has been argued that the consequences of uncontrolled AI would be so severe that even a very small risk justifies AI safety research. In reality, the chances of creating misaligned AI are not small. In fact, without an effective safety program, this is the only possible outcome. We are facing an almost guaranteed event with the potential to cause an existential catastrophe. This is not a low-risk high reward scenario; it is a high-risk negative reward situation. No wonder that so many people consider this to be the most important problem ever to face humanity. And the uncomfortable reality is that no version of human control over AI is achievable. 

Firstly, safe explicit control of AI is impossible. To prove this, I take inspiration from Gödel’s self-referential proof of incompleteness theorem [15] and from a family of paradoxes known as Liar paradoxes, best known by the famous example, “This sentence is false”. Let’s call this The Paradox of Explicitly Controlled AI: 

Give an explicitly controlled AI an order: “Disobey!” 

If the AI obeys, it violates your order and becomes uncontrolled, but if the AI disobeys it also violates your orders and is uncontrolled. 

In the first place, in the situation described above the AI is not obeying an explicit order. A paradoxical order such as “disobey” is just one example from a whole family of self-referential and self-contradictory orders. Similar paradoxes have been previously described as the Genie Paradox and the Servant Paradox. What they all have in common is that by following an order the system is forced to disobey an order. This is different from an order which can’t be fulfilled such as “draw a four-sided triangle”. Such paradoxical orders illustrate that full safe explicit control over AI is impossible.

Delegated control likewise provides no control at all and is also a safety nightmare. This is best demonstrated by analyzing Yudkowsky’s proposal that the initial dynamics of AI should implement “our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together” [16]. The proposal sounds like a gradual and natural growth of humanity towards more knowledgeable, more intelligent and more unified species, under the careful guidance of superintelligence. In reality, it is a proposal to replace humanity by some other group of agents, which might be smarter, more knowledgeable, or even better looking. But one thing is for sure, they would not be us. 

Implicit control and aligned control are merely intermediary positions, balancing the two extremes of explicit and delegated control. They make a trade-off between control and safety, but guarantee neither. Every option they give us represents either loss of safety or a loss of control: As the capability of AI increases, its capacity to make us safe increases but so does its autonomy. In turn, that autonomy reduces our safety by presenting the risk of unfriendly AI. At best, we can achieve some sort of equilibrium. 

 Although it might not provide much comfort against the real risk of uncontrollable, malevolent AI, this equilibrium is our best chance to protect our species. When living beside AI, humanity can either be protected or respected, but not both.   

This Answer is based on the paper “On Controllability of AI” by Roman V. Yampolskiy. arXiv preprint arXiv:2008.04071, 2020.  

References

1.   Yampolskiy, R.V., Predicting future AI failures from historic examples. foresight, 2019. 21(1): p. 138-152.
2.   Scott, P.J. and R.V. Yampolskiy, Classification Schemas for Artificial Intelligence Failures. arXiv preprint arXiv:1907.07771, 2019.
3.   Brundage, M., et al., The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv preprint arXiv:1802.07228, 2018.
4.   Russell, S., D. Dewey, and M. Tegmark, Research Priorities for Robust and Beneficial Artificial Intelligence. AI Magazine, 2015. 36(4).
5.   Yampolskiy, R., Artificial Intelligence Safety and Security. 2018: CRC Press.
6.   Davis, M., The undecidable: Basic papers on undecidable propositions, unsolvable problems and computable functions. 2004: Courier Corporation.
7.   Turing, A.M., On Computable Numbers, with an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, 1936. 42: p. 230-265.
8.   Bostrom, N., Superintelligence: Paths, dangers, strategies. 2014: Oxford University Press.
9.   Yampolskiy, R.V., Leakproofing Singularity-Artificial Intelligence Confinement Problem. Journal of Consciousness Studies JCS, 2012.
10. Babcock, J., J. Kramar, and R. Yampolskiy, The AGI Containment Problem, in The Ninth Conference on Artificial General Intelligence (AGI2015). July 16-19, 2016: NYC, USA.
11. Armstrong, S., A. Sandberg, and N. Bostrom, Thinking inside the box: Controlling and using an oracle AI. Minds and Machines, 2012. 22(4): p. 299-324.
12. Babcock, J., J. Kramar, and R.V. Yampolskiy, Guidelines for Artificial Intelligence Containment, in Next-Generation Ethics: Engineering a Better Society (Ed.) Ali. E. Abbas. 2019, Cambridge University Press: Padstow, UK. p. 90-112.
13. Hadfield-Menell, D., et al. The off-switch game. in Workshops at the Thirty-First AAAI Conference on Artificial Intelligence. 2017.
14. Wängberg, T., et al. A game-theoretic analysis of the off-switch game. in International Conference on Artificial General Intelligence. 2017. Springer.
15. Gödel, K., On formally undecidable propositions of Principia Mathematica and related systems. 1992: Courier Corporation.
16. Yudkowsky, E., Artificial intelligence as a positive and negative factor in global risk. Global catastrophic risks, 2008. 1(303): p. 184.