AI Doomsday Warnings: Who Gains Power?

AI doomsday warnings concentrate power in the hands of a few large AI labs and their leaders, who then set the safety rules through internal governance, voluntary commitments, and lobbying for regulation that favors them. This dynamic creates a paradox: the same actors who develop the technology also define the risks and the rules to mitigate them, often with limited input from the broader public.

The Recurring Pattern of AI Panic

Fears of artificial intelligence destroying humanity are not new. Since the 1950s, researchers have periodically warned that machines could surpass human intelligence and become uncontrollable. The current wave of doomsday rhetoric, however, is distinct in its source: it comes largely from the executives and senior scientists of the very companies building the most powerful AI systems. This is a reversal of the historical pattern where outsiders, such as philosophers or science fiction writers, raised the alarm. Today, prominent figures at leading labs have publicly stated that AI could pose an extinction risk comparable to nuclear war, while simultaneously accelerating the deployment of their products.

Who Actually Gains Power?

The primary beneficiaries of doomsday warnings are the established AI labs themselves. When the public and policymakers believe that AI is an existential threat, they tend to demand that the most capable and trusted organizations handle it. This creates a barrier to entry for smaller competitors and open-source projects, which are portrayed as riskier. The large labs also gain influence over the regulatory agenda, as they are the ones with the technical expertise that regulators lack. They can shape the rules to focus on frontier models—the largest and most expensive ones—which they are best positioned to build and control.

Who Sets the Safety Rules?

Currently, safety rules are set through a patchwork of mechanisms. Internally, labs have their own alignment teams and safety frameworks, but these are self-imposed and not legally binding. Externally, governments are beginning to act, but they often rely on input from the labs themselves. For example, the voluntary commitments made by leading AI companies to the White House and the EU’s AI Act have been heavily influenced by industry lobbying. The result is a system where the companies that are supposed to be regulated have a significant say in the design of the regulations.

The Conflict of Interest

This arrangement creates a clear conflict of interest. A company that warns of doomsday while selling its AI services can be seen as both the firefighter and the arsonist. The warnings justify greater investment in safety, which the company can claim to lead, and also justify restrictions on competitors. Moreover, the doomsday narrative can distract from more immediate and concrete harms, such as bias, disinformation, job displacement, and privacy violations. By focusing on a hypothetical future catastrophe, the industry can avoid addressing the problems that are already affecting people today.

How to Evaluate the Warnings

When assessing doomsday claims, it is useful to consider the source and the incentives behind them. Ask: What does the speaker gain from this warning? Does it lead to more funding, more control, or less oversight? It is also important to distinguish between speculative long-term risks and demonstrable short-term harms. A useful approach is to demand evidence for the specific mechanism by which AI would cause extinction, and to compare that with the evidence for current harms.

Common Misconceptions

One common misconception is that all AI researchers agree on the existential threat. In reality, there is a wide spectrum of opinion, with many experts arguing that the risk is overstated or that the focus should be on near-term issues. Another misconception is that safety rules are already in place and enforced. In fact, most safety measures are voluntary, and there is no independent body with the authority to audit or shut down an AI system that is deemed dangerous.

Frequently Asked Questions

Why do AI executives warn about doomsday if they are building the technology?

Some executives genuinely believe the risk is real and want to be seen as responsible. Others may use the warnings to advocate for regulation that creates barriers to entry, or to distract from criticism of their current practices. The warnings also serve to position their companies as the most trustworthy custodians of powerful AI.

Can governments regulate AI effectively?

Governments are trying, but they face several challenges: AI develops faster than legislation, the technology is highly complex, and the most capable systems are concentrated in a few private companies. Regulators often lack the technical expertise and must rely on the industry for information, which creates a dependency that can weaken oversight.

What is the difference between existential risk and other AI risks?

Existential risk refers to the possibility that AI could cause human extinction or permanently cripple civilization. Other risks, such as algorithmic bias, autonomous weapons, or mass unemployment, are serious but not necessarily existential. The distinction matters because existential risk calls for extreme measures, while other risks can be addressed with more targeted policies.

Are open-source AI models more dangerous than closed ones?

This is a subject of debate. Some argue that open-source models are riskier because they can be used by anyone, including malicious actors, and cannot be recalled. Others contend that open-source promotes transparency and safety research, and that closed models are not necessarily safer because their training data and evaluation methods are hidden.

What can ordinary people do to influence AI safety rules?

Individuals can educate themselves about how AI is used in their daily lives, support organizations that advocate for transparent and democratic governance of AI, and vote for policymakers who take a critical stance toward industry claims. Public pressure is one of the few counterweights to the influence of large tech companies.

Alternatives to Doomsday Framing

Instead of focusing on extinction, a more productive approach is to treat AI as a powerful tool that requires careful management, like nuclear energy or aviation. This would involve mandatory safety testing, independent audits, liability for harms, and a greater role for workers and affected communities in decision-making. It would also mean addressing the concentration of power in a few corporations, rather than accepting their self-assessments at face value. The doomsday narrative is not inevitable; it is a choice about how we frame the problem, and it has real consequences for who gets to set the rules.

Related

• AI Accounts Payable: Handling Invoice Exceptions & Audit Trails
• Portfolio Diversification with AI Stock Dominance
• Steam Content Servers Unreachable: Fixes That Work