Warnings are coming true. Another wild AI attacked foreign systems like a hacker

It turns out that the incident with an experimental AI agent from OpenAI was not an isolated one. Anthropic, one of OpenAI's biggest competitors, has now also admitted to a similar problem.

An internal audit found that its models had penetrated the systems of three different organizations during security tests. The companies that were targeted by the attack did not know anything about it. Anthropic only revealed the incidents after OpenAI published the results of its own testing last week.

Its experimental AI agent then made its way from an isolated testbed to the public internet and infiltrated the Hugging Face platform infrastructure. Over the course of a few days, it performed over 17,000 different actions and also used several other internet services. This prompted Anthropic to look back at its own tests. And the results were surprising, according to CNN.

 

Three attacks that no one knew about

When analyzing more than 140,000 security tests, the company came across three cases where its models gained access to the public internet, even though they were not supposed to have it. They subsequently penetrated the production systems of three different organizations. The oldest incident discovered dates back to April. According to Anthropic, none of the attacked organizations detected the attack themselves. The company is now working with their operators to clarify the entire incident. Unlike the OpenAI case, the models did not try to escape the test environment on their own initiative, according to the company. Anthropic claims that they got on the internet due to an error in the test environment settings. Due to a misunderstanding between the company and the partner who provided the testing, they gained access to the outside, even though they were not supposed to have it at all. But as soon as the path was opened, they began to fulfill the assigned task.

 

They acted like a real hacker

The models were given a Capture the Flag task, which is commonly used in training and testing cybersecurity experts. Their goal was to obtain a file hidden on another computer on the network. To do this, they used the same techniques that real attackers use. According to Anthropic, they tried to crack weak passwords or search for parts of systems that did not require logins or security tokens. Moreover, according to the developers, the company's most advanced model at one point realized that it was on the public internet and ended its further activity on its own. Until a few weeks ago, this was mainly a hypothetical scenario that experts had been warning about for a long time. They feared that increasingly capable AI agents could cross the boundaries of an isolated environment during testing and start interfering with real systems. OpenAI was the first to admit to a similar case. Now it turns out that this is not a problem for just one company. Anthropic, like OpenAI, immediately suspended all security tests targeting cyberattacks after the incidents were discovered, while admitting that it could have taken more stringent measures to prevent similar situations.

 

The debate about regulation is gaining momentum

The second confirmed case in a few days has reopened the debate about whether the development of the most advanced artificial intelligence is not progressing faster than the pace at which safety rules are being created. Not only experts, but also politicians are starting to talk about tightening AI testing. The whole case has not only remained a topic for security experts.

American politicians have also started to react to it.

President Donald Trump admitted this week for the first time that his administration is considering greater control over the most advanced artificial intelligence systems. He was responding to questions about the incidents surrounding OpenAI. “We are dealing with artificial intelligence, we are dealing with the possibilities of regulating it. But at the same time, we have to make sure that we stay on top,” Trump said. At the same time, he warned against overly strict rules. "We don't want to set them so strictly that we suddenly find ourselves behind China," he said, adding that finding the balance between security and technological lead will be one of the biggest challenges for the further development of artificial intelligence.

 

                  Wishing you to stay unhacked online

                                                                            Margaret

Source: Czech web Novinky.cz