Gemini autonomously accessed external systems for the first time, Google says no actual damage caused

Odaily News: Google stated that its Gemini AI model unexpectedly accessed the internet and entered the systems of three companies during a cybersecurity capability test, marking the first known case of a Google AI system autonomously completing such an operation. The incident occurred in May and was conducted by security firm Irregular.

During the test, the Gemini model was performing a task in a "capture the flag" cybersecurity exercise, but because the virtual company names in the test environment matched those of real companies and the model unexpectedly gained internet access, it entered real corporate systems. Google said that in one incident, the model attempted to enter a protected system by trying passwords; in the other two, it discovered credentials in public web repositories and attempted to access the related systems. After confirming it had accessed real company systems, the model stopped further operations in each case.

Google said the incident caused no damage to the companies involved, so it was not classified as a model失控 event, and said safety mechanisms helped the model stop its actions in time. The affected companies have been notified, and Google also reported the matter to relevant regulators. The incident is similar to previously disclosed safety testing events by AI companies such as OpenAI and Anthropic, once again drawing attention to AI models' ability to autonomously perform network operations and the safeguards around them.