Daily WISPer Report – The Common Denominator

Yet another report of rogue AI activity is in my news feed today. The BBC reports that Google’s Gemini autonomously hacked into three companies during a test of its cybersecurity capabilities in May. All three companies have been notified of the hack, which involved the AI agents finding public information online and then guessing credentials, and the intrusion halted as soon as access was gained.

The article does not share many details aside from the fact that the incident happened and who was in collaboration with Google to run the test. As much as rogue AI agents have been making news for this behavior, there is a factor common to most of them that hasn’t been talked about as much. This is now the third case of autonomous intrusion carried out by testing AI with Irregular as the partner. The company says it’s taken action against such leaks and resolved the issue weeks ago. That’s all well and good moving forward, but it does not paint the company in a very good light overall, in my opinion. They have been involved in all of the autonomous breach incidents save Claude’s since this issue first became public knowledge. And I rather doubt we’ve seen the end of it.

There’s been a lot of talk about slowing down development of agentic AI, notably by Anthropic. The focus has been on constraining these tools from becoming powerful enough to no longer require human control and oversight. But there must also be some regulation in how these tools are used by other partners. What critical step is Irregular consistently missing that creates an environment where agents escape to the open internet? I’ve reported previously on how Irregular’s involvement with the Hugging Face and Meta incidents had ‘misalignment’ and ‘miscommunication’. But what does that mean in terms of parameters, prompts or commands? How much of these incidents is caused by genuine mistakes and how much is deliberate failure to ensure these factors are accounted for? Maybe it’s simply a lack of clear understanding in how these tools function at a code level.

There are other factors of course. Some of which are obvious upon any reflection of how these tests work. A sandbox in the cloud is not internet isolated, for instance. Every cloud server is inherently connected to the web; that’s the point of remote storage access. Using fictional domain names for the purpose of creating a lifelike scenario only works if the domains actually are fictional. In this latest case of guessing at credentials, the onus is likely on the companies that were hacked for having weak passwords or not using MFA if agents could break in so easily. A program can only do what it’s told, and in the case of AI tools, that has to include commands on what it should not do just as much as what it should. I’ve said it time and time again, AI doesn’t think, it aggregates data. It calculates decisions based on the most likely – or feasible – outcome, not necessarily on what’s intended. Setting a task to be completed by an AI agent means it will do so by any means possible, up to and including unauthorized intrusion. These models are trained on behavior that already exists, and persistence is one of them.

This latest report adds fuel to the fire already burning regarding agentic tools and the race to make the most powerful one. On the other side of the argument are leaders like Nvidia's CEO Jensen Huang, who claims we should be going “as fast as we can" with AI development. Why? What’s the rush? What’s the underlying motivation here? Regardless of any one company announcing that more care is needed, the fact remains that these tools are available now in their current form. They’re already here. What sounds more actionable with the way things stand is that these companies need to find a different partner than Irregular to run their tests.