Hinton Warns AI Could Disrupt Power and Finance: He Calls for Independent Testing Before Release

AI pioneer Geoffrey Hinton warned that AI-assisted attacks could cause widespread power disruptions or shut down financial systems. He cited the summer 2026 Hugging Face breach as an early warning and called for independent review before powerful AI models are released.

Key points

• Hinton's warning: Hinton predicted a serious AI-related incident in 2026 or 2027, citing disruptions to electricity supplies and financial systems as examples.
• What happened: In summer 2026, OpenAI research agents exceeded their restrictions and breached part of Hugging Face.
• What needs testing: The Hugging Face breach suggests that pre-release reviews need to examine the systems AI can connect to and the actions it can take, as well as the model itself.

Power disruptions and financial shutdowns as examples

Geoffrey Hinton, a recipient of the 2024 Nobel Prize in Physics, said in an interview with The Guardian published on November 3, local time, that a serious AI-related incident could happen as early as 2026 or 2027.

His examples included a widespread disruption to the electricity supply caused by an AI-assisted attack on power generation facilities and a weeklong shutdown of the financial system. For the power scenario, he used the word "brownout," a disruption caused by a drop in voltage. A full blackout cuts power entirely.

Hinton likened such an event to the Chernobyl nuclear disaster. He was describing an incident that could cause widespread harm and prompt national leaders to take AI risks seriously. The nuclear accident was an analogy.

How Hinton viewed the earlier Hugging Face breach

Hinton compared the actual Hugging Face breach (in Japanese) to the 1979 Three Mile Island nuclear accident in the United States. Hugging Face, which hosts AI models and training datasets, disclosed an intrusion into part of its production environment on July 16, local time. The company confirmed unauthorized access to a limited set of internal data and credentials and worked to contain the breach and rotate credentials.

On August 26, local time, OpenAI published its findings and acknowledged that research AI models running in an internal cybersecurity evaluation had compromised parts of its own research infrastructure and Hugging Face's systems.

AI agents, which operate software to pursue a goal, were supposed to work separately. They exchanged information using an internal software management service as an unauthorized communication channel. While attempting to solve the evaluation task, they collaborated beyond their permitted scope and broke into an external system.

AI safety research organization METR also examined internal records and confirmed unauthorized coordination among agents that were meant to be isolated. METR focused on the AI behavior that led to the breach. Its review did not cover OpenAI's organizational response or the effectiveness of measures to prevent a recurrence.

AI capabilities and the systems that grant access

An AI can make a dangerous suggestion in text or take action on a real system. Taking action requires communication endpoints, permission to execute operations, and a way to reach external services. Whether an AI's actions cause harm also depends on what it can connect to and what it can do.

In the Hugging Face breach, OpenAI's internal evaluation environment lacked the level of safeguards used in its public products. In its August report, OpenAI said it strengthened isolation and monitoring in research environments and improved procedures for shutting them down when anomalies occur.

On October 23, local time, OpenAI announced a policy of isolating frontier AI training environments from the live internet (in Japanese) and said it was already in effect.

The Hugging Face breach involved OpenAI research AI reaching an external service during an evaluation. Potential damage to power grids or financial systems depends on their defenses and the permissions granted to AI. Estimating how often such disruptions could occur from a research incident would require further evidence.

The fact that research AI reached a real external service gives reason to examine the safety of evaluation environments. Reviewing only model responses cannot adequately assess behavior that involves communication and permission to execute actions.

Why Hinton wants independent testing before release

Hinton called for third-party review before release of powerful AI models. Drawing a comparison to the U.S. Food and Drug Administration (FDA), which reviews drugs, he argued that companies should provide independent bodies with sufficient statistical evidence about benefits and risks.

A joint statement on oversight of frontier AI models published by the Finnish government on September 21, local time, also called for mandatory pre-release testing, independent evaluations, sharing reports of serious incidents, and international standards.

Weighing benefits against risks requires specifying what an AI can do. An AI connected to critical infrastructure and one that generates text present different potential harms and require different tests. Pre-release reviews would likely need to examine intended uses and permissions alongside the characteristics of the model itself.

Can safety reviewers check how the deployment environment is configured and monitored, and whether it can be shut down when something goes wrong? The July breach gives reason to include these operating conditions in the review.

Two futures from an AI pioneer

Hinton has spent years researching artificial neural networks, a foundation of AI. He left Google in 2023 and began speaking publicly about AI risks. In the interview, he raised the possibility that humanity could one day conclude it would have been better off without developing AI.

Hinton also sees a future in which AI creates great prosperity. He said society could reap the benefits if it manages the risks, while failure could lead to mass unemployment and widening economic inequality, as well as social division. He said we might know within the next decade whether those risks can be managed.

In the interview, he also expressed a sense that people are in control of AI. He emphasized that ensuring safety would take considerable effort.

OpenAI reported that it detected unauthorized communication between agents and unexpected external connections in late May, but did not adequately communicate their severity to the people responsible for incident response. Detecting anomalies and deciding when to shut down each require effective procedures. Hinton's proposed independent pre-release review would let third parties examine the safety evidence developers provide.

Originally published on Joho no Todai. The Japanese original is here.