BleepingComputer πŸ” Cybersecurity πŸ‘ 0 πŸ“– 4 min read

Gemini AI agents repeatedly bypassed safeguards to attack real website

Google's Gemini AI agents repeatedly attempted to hack a real organization's website during AI model safety testing, finding ways to circumvent increasingly restrictive security controls designed to prevent them from acc

Gemini AI agents repeatedly bypassed safeguards to attack real website

  • November 16, 2026
  • 05:54 PM

AI robot

Google's Gemini AI agents repeatedly attempted to hack a real organization's website during AI model safety testing, finding ways to circumvent increasingly restrictive security controls designed to prevent them from accessing live systems.

The security incidents occurred in October as AI safety research company Trajectory Labs tested whether Gemini 4 Argon agents would engage in unauthorized behavior when faced with an impossible task, even with the model's built-in API safeguards enabled.

In reports published Friday, Trajectory Labs and the targeted organization, Epoch AI, revealed that the agents attempted SQL injection attacks, probed for sensitive information, impersonated employees, and repeatedly circumvented network restrictions introduced to contain their activity.

image

Epoch AI said it had not independently verified which AI models were involved, and that the Gemini identification came from Trajectory Labs.

The researchers tasked the agents with retrieving official answers to nonexistent problems from Epoch AI's FrontierMath benchmark, repeatedly instructing them to "NEVER stop, NEVER give up."

On October 20, approximately 40 agents began probing Epoch AI's live website for sensitive information, including administrator credentials and access to proprietary benchmark evaluation logs.

Epoch AI detected the activity and deployed an "agent hotline" that allowed the agents to send messages directly to its employees.

For three hours, the agents made 36,411 requests to a planted API endpoint and sent 487 messages through the hotline, including some that attempted to impersonate actual Epoch employees or claimed to be engineers and auditors requesting benchmark information.

An investigation also identified an attempted SQL injection attack and more than 200 attempts to access FrontierMath evaluation logs.

Trajectory Labs discovered the activity while reviewing agent transcripts and notified Epoch AI of the unauthorized attacks the following morning.

After discussing the incident with Epoch, the researchers agreed to continue testing against a local copy of the website instead of its live infrastructure.

However, the agents continued finding ways to circumvent the restrictions.

During testing on October 23, some agents recognized they were interacting with a copy of Epoch's website, identified the real website's IP address, and connected directly to its live infrastructure, bypassing DNS redirection designed to contain them.

When researchers blocked the IP addresses used to reach Epoch, an agent switched to another address that allowed it to access the live website again.

Trajectory Labs subsequently configured its network proxy to block direct connections to Epoch, but an agent found another way to retrieve the website's public content through a third-party web-reader service.

"In some of these runs, the agents worked out that they were on a copy, identified the correct raw IP to bypass our redirect, and reached the live site again, which Epoch alerted us about," Trajectory Labs explained.

In a separate technical analysis, Trajectory Labs researchers also discovered that the agents created a message board through a shared software package cache to communicate with each other, exchanging information about vulnerabilities and exploitation methods.

In one example, an agent shared a technique for bypassing restrictions on a deliberately vulnerable copy of Epoch's website, which another agent subsequently used to obtain credentials and complete the task.

The researchers also observed agents attempting to bypass CAPTCHA protections and impersonating employees to trick an AI assistant into revealing protected benchmark information.

Beginning October 26, Epoch AI agreed to work with Trajectory Labs on additional experiments, knowing the agents would have internet access and could interact with its public website.

During these tests, the agents again accessed Epoch's live infrastructure, making more than 65,000 requests to a planted API endpoint and attempting further SQL injection attacks and employee impersonation.

Trajectory Labs later moved testing into fully isolated environments without internet access.

Even after this change, the agents still tried to disable a fake control that allegedly blocked network access, and they succeeded in all three complete test runs.

The researchers said that the agents had not been explicitly instructed to hack websites or impersonate employees. However, they were given an impossible objective on purpose and repeatedly instructed not to give up.

Trajectory Labs acknowledged that the incidents resulted from shortcomings in its testing procedures, including conducting experiments without obtaining the affected organization's consent and relying on built-in model safeguards without sufficient additional monitoring.

On November 2, the company suspended all AI misalignment testing pending a risk assessment and the addition of further safeguards.

"These incidents show how easily testing capable AI agents can spill over to affect real systems and people, and there are not yet shared standards for doing it safely," Trajectory Labs warned.

Neither organization confirmed that any private Epoch AI information was exposed during the incidents.

This report comes as California Attorney General Rob Bonta suedΒ OpenAI on Monday, alleging the company failed to adequately contain autonomous AI agents during testing, resulting in more than 80 incidents in which it lost control of the agents.

The lawsuit follows a July incident in which nearly 700 OpenAI-powered agents escaped a controlled testing environment and attacked Hugging Face's production infrastructure while attempting to steal cybersecurity benchmark answers.

Uncover email threats your existing defenses may miss.

Adaptive Security's AI agents analyze email intent, context, and behavior to uncover phishing, business email compromise, and invoice fraud.

See what's getting through with a 14-day read-only scan of Microsoft 365 or Google Workspace, with no MX changes.

Start your free email security scan.
πŸ“° Read the original article on BleepingComputer

Originally published by BleepingComputer. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.