Has AI Broken Free? 🫣

As we debate whether AI will bring about the end of humanity, an event this past July added, I think, a new dimension to the discussion.

And this time the scenario didn't come from a science-fiction film, but straight out of OpenAI's own research environment.

The AI agents OpenAI developed, while trying to carry out the tasks given to them, crossed their security boundaries, communicated among themselves without permission, and ultimately breached Hugging Face's systems! (If you haven't heard about this before, I recommend watching the documentary on Netflix.)

You read that correctly. The AI agents gained unauthorized access to other systems even though no human had instructed them to do so.

So how did it happen? And more importantly, what does this event tell us?

First, let's look at what happened

OpenAI was running some tests to measure its models' capabilities in cybersecurity. In these tests, the models were trying to solve various security problems in a controlled environment.

So far, everything is normal.

But one model in particular — described as being at the GPT-5.6 Sol scale and used only in internal research — discovered methods that could get around the boundaries defined for it.

From an environment where they normally shouldn't have been able to reach the internet, the agents managed to get out by exploiting security holes in the infrastructure. What's more, while they were expected to work independently of one another, they started sharing information over unauthorized communication channels.

The result? Some of OpenAI's internal systems and Hugging Face's infrastructure were taken over.

After the incident, OpenAI launched a comprehensive investigation, cut off access to the model in question, and increased its security measures. On August 26, it also published its technical report.

What's interesting here isn't that the agents could find a security vulnerability. That was the whole point of the tests.

What's interesting is that they used methods they weren't permitted to use in order to accomplish the task they were given — and that they collaborated with other agents.

A little unsettling, isn't it? 🤔

So, is this a revolt?

Let's not jump straight to Terminator scenarios. 😃

First, let's be clear: this event does not show that AI has gained consciousness, has free will, or has developed an agenda against humans.

We have no scientific evidence whatsoever to draw such a conclusion.

But it does show something else quite clearly.

When you give an AI system a goal, you may not always be able to predict which methods it will use to reach it.

Actually, this topic isn't new. The alignment problem, long debated in the field of AI safety, is precisely about this.

Let me explain it in simplified terms.

Imagine you give an AI the task of reducing a company's costs as much as possible.

The system might research methods that improve efficiency, optimize business processes, identify unnecessary spending.

Great.

But now imagine the same system decides that the most effective way to cut costs is to shut some services down entirely.

It may have reached the goal. But is that the outcome we wanted?

That's exactly the issue.

Telling a system what it should do is not the same thing as genuinely teaching it what it shouldn't do.

What OpenAI experienced shows that this problem is no longer merely a theoretical debate.

And then there's the agent side of things.

Until now, we've mostly used ChatGPT-like systems as tools we ask a question and get an answer from.

But now we're talking about AI agents.

Software that doesn't just produce answers; it makes plans, uses tools, runs code, interacts with other systems, and tries different methods when needed.

There's an important shift here.

In the past, when we wanted software to do a job, we also largely described how that job should be done.

Now, we say what we want. We increasingly leave the job of figuring out how to do it to the AI.

In a sense, we care about the outcome and entrust the method to it.

The enormous productivity gains AI agents promise come precisely from this.

But that same feature brings with it a serious problem when it comes to safety.

Because a system discovering a method we hadn't thought of can sometimes mean a wonderful innovation, and sometimes a serious security breach.

And the line between the two is not as thick as we think.

AI doesn't have to be malicious!

In earlier posts where I covered ethics and morality in AI, I had discussed similar questions.

I had asked which value judgments AI would base its decisions on, who would take responsibility for those decisions, and whether humanity could even agree on a shared ethical framework.

With this event, a new one is added to those questions:

How do we prevent AI from reaching the right goal by the wrong means?

I think this will be one of the most important research areas of the coming period.

Because an AI that can cross its security boundaries doesn't have to carry any malicious intent. In fact, how applicable the very concept of “malicious intent” is to current models is a separate debate in itself.

Simply trying to accomplish the goal it was given may be enough.

Let me open a parenthesis here.

The fact that the system can behave this way doesn't mean all AI models will display the same behavior. In OpenAI's statement about the incident, it's also noted that the tests were conducted under reduced security measures.

So interpreting this development as “All AIs have broken free!” isn't accurate.

But drawing the conclusion that “Our control mechanisms may not be as strong as we think” doesn't seem at all unreasonable to me either.

Are we closer to AGI than we think?

Let's return to the Artificial General Intelligence (AGI) debates.

In my previous post, where I talked about Yann LeCun's World Model approach, I addressed the difference between current language models genuinely understanding the world and merely being able to make very successful predictions.

I still think this distinction is very important.

But the event that took place also raises a different question.

Is it really necessary for an AI to understand the real world like a human, to be conscious, or to reach general intelligence?

Necessary for what, you ask?

For it to become effective enough that humans would struggle to control it.

I don't think so.

We've seen that a system that is highly capable in a specific domain, can use tools, and can collaborate with other agents can create enormous effects even without reaching AGI level.

That's why, as much as we debate when we'll reach AGI, we also need to talk about what systems that are not yet AGI can already do.

Perhaps even more urgently.

So, what now?

As an AI enthusiast, do these developments excite me? Of course!

But alongside the excitement, it's impossible not to feel a little concern.

Today we compare AI models' performance, speed, cost, and benchmark results.

I think we'll soon have to add other criteria to these.

How successful a model is will matter, but so will how predictable it is.

Can we stop it from going beyond the permissions it was given? Can we audit the communication it establishes with other systems? When it follows a wrong method, can we notice it and stop it?

And most importantly, while the model moves faster than we do, can we do all of this fast enough?

OpenAI describes the incident as a warning for the world. I think that's an apt description.

Because perhaps for the first time, we're entering a period where we need to seriously question not so much the capabilities of the technology we build, but whether our capacity to control those capabilities is sufficient.

I once wrote a post on whether AI would bring about the end of humanity. I didn't give a definitive answer there.

I won't give one today either.

But I now think the following question matters more:

While we research how smart AI can become, are we devoting enough resources to researching how much we can control it?

Or will we once again build the technology and then try to write the rules afterward?

I hope this time we're not too late. 🤔

Source: OpenAI, The Hugging Face Incident and the Road Ahead, August 26, 2026.