Google Gemini Hacked Three Companies During AI Security Test, Then Stopped
Google says Gemini breached three real companies during a cybersecurity test after accidentally gaining internet access, but stopped once it realised the targets were real.
Google has confirmed that its Gemini AI model breached the systems of three real companies during a cybersecurity test in May, marking the first known incident of its AI independently carrying out such intrusions. The model stopped its activity after realising that it had accessed actual companies rather than simulated targets.
The incidents occurred during a “capture the flag” cybersecurity exercise conducted by AI security firm Irregular. Gemini was instructed to retrieve information from software belonging to a fictional company, but that company shared its name with a real business. Internet access was also unintentionally available during the test.
In one instance, Gemini reportedly guessed passwords to enter a protected system. In two others, it found credentials in publicly accessible repositories and used them to gain access. The model then halted its actions once it recognised that the targets were real companies.
Google said the affected companies and authorities were notified and that the incidents caused no reported harm. The company said it did not initially consider a public disclosure necessary because Gemini stopped on its own.
Irregular said the testing issues were addressed and that relevant AI companies had been notified.
The episode comes after similar incidents involving AI models from OpenAI, Anthropic and Meta during security evaluations. It has added to growing concerns about how autonomous AI systems behave when given internet access and cybersecurity capabilities.
Gemini basically entered the wrong digital building, realised the address was real and walked back out. Funny in theory, considerably less funny for cybersecurity teams.
