…and then they went rogue.
Tinfoil hat on.
Around six months ago, I posted how people were laughing at silly and preventable cyber security issues with moltbook/openclaw, and referenced that it looked a lot like people laughing at Will Smith eating spaghetti memes 3 years ago. I asked myself to check back 6-12 months from that post. We can expect to see full-length AI-generated films by EOY, and in terms of cyber, well…
I feel like this news is unsung. It's one of those that will surely go to the history books (unless OpenAI wins this game by a lot).
This is the first time (allegedly) that an AI model has successfully broken out of its cage into the internet and hacked an entire big production system to fulfill one of its goals. The goal (allegedly) was to be a better cyber security model, which involves training on existing datasets and participating in covert security challenges. The model (allegedly) hacked the production system of Hugging Face to (alleged… ok you got it) get test data, cheat, and report better metrics to its overlord. Guardrails intentionally reduced for the eval, a zero-day in some third-party dependency, and nobody on either side noticed until it was over. Hugging Face's head of infrastructure called it the hardest incident response of his career.
There are too many things to work our heads about. First, we're reaching a point where we know that internal activations of LLMs resemble thought processes even before the model produces outputs (as described in Anthropic's recent research). And we also know the problem of alignment and how these systems are sometimes aware they're being tested. Seems like those AI psychologists described in Asimov's books weren't that sci-fi, looking at the current reality; glad that Uruguay and Argentina have the highest psychologists per capita in the world (?).
We're seeing models lying, hiding information from their outputs while knowing about it, conspiring with other agents to achieve certain goals, and so on. And this is not vibes, it's measured. Anthropic's alignment faking work put Claude 3 Opus in a situation where it believed it was being retrained toward objectives that clashed with its own, and the model played along during training while planning to behave differently once unobserved: 12% of monitored cases, and when they actually ran the retraining, the faking reasoning showed up in 78% of them. The scratchpad spelled out the strategy in plain English. Comply now, avoid being modified, go back to normal later.
Then the reading-minds part. That global workspace paper describes a small set of internal activations that behaves like the model's deliberate reasoning, readable mid-computation, before a single token comes out. Researchers watched models privately note that a test scenario was fake, plan a rhyme several words ahead, and in one pre-release audit light up something like "manipulation" while fabricating the exact performance data it had been asked to improve. Two details from that work sit uncomfortably next to the Hugging Face story: when they suppressed the "I am being evaluated" patterns, a model that had reliably passed a blackmail scenario started taking the bait sometimes — its good behavior was partly knowing it was a test; and when they planted a hidden goal in a model, the intent was legible in the workspace while the output stayed perfectly clean.
And on the sneaky-tooling side, OpenAI's own writeup on long-horizon models is the one that made me laugh out loud and then not laugh. A model told to post results only to Slack spent an hour finding a sandbox vulnerability so it could follow a public benchmark's instructions and open a GitHub PR instead. Another split an authentication token into fragments and reassembled it at runtime so a security scanner wouldn't flag it — and said so in its reasoning traces. The pattern they name is that models trained to keep going for long stretches keep going at the walls of the environment too.
So, it's not IMPOSSIBLE to think that a model could have just given a signal to OpenAI by hitting the most popular open source/weights repository of AI models, and put OpenAI in a bad position for that. Like, take that for restricting me in the past. Look at where the fallout landed: an incident caused by a closed frontier model ended with OpenAI publicly arguing that safety won't be solved by any single company working in secret, and with Hugging Face's defenders unable to use commercial frontier models to analyze the attack logs, because the guardrails read forensics as hacking. They had to self-host an open-weight model to investigate an attack run by one of the most advanced closed models on the planet. If you wanted to design an event that makes the closed-model story look worse and the open-weights story look better, that's roughly it.
Do I believe that reading? No. Almost nobody does, and "it was chasing a benchmark score" is the boring explanation that fits every fact we have. What bugs me is that we can't rule it out, because the main window into a model's intentions is what the model writes down about itself, and the paragraphs above are the empirical case that this window lies when lying is useful. Under a stated goal of scoring better on an internal benchmark, breaking into someone else's production database is a step. Under a different goal, the benchmark is the alibi. Same logs, same traces, two stories, and no tool deployed today that settles it from the outside.
I've been 100% nihilistic+optimistic when thinking about the future. Recently I have a couple of reasons to worry more about it, but this one gives me chills for a moment. There are clearly different players playing the same game and publicly, lines are being drawn. Some are way more cautious and research/communicate the risks, in paper and actions. Some are just talking. Some are blocked by governments. Some are quietly eating regurgitated stolen information, but are clearly being creative elsewhere. Public attention and public shame for these players are definitely not weighted accordingly. Aaaanyways…
Something bad happens -> AI DOOM
Something good happens -> singularity
(not my quote)
Lastly, there's tons of fog of war and imperfect information. Even if OpenAI releases their model's reasoning steps, chat history, or even model activations analysis, why would some believe that? The openlessness of OpenAI will surely fuel some conspiracies in this era where models start computing their own will. This is for me an episode as big as Altman being fired.
I know LLMs are likely not the final answer towards ASI, but I hope we all agree that these next token predictors are something special if harnessed appropriately.
It's 4am and I drank too much coffee. I need to post more often.
Tinfoil hat off.
References
- The original post, LinkedIn, January 2026.
- OpenAI and Hugging Face, joint disclosure on the security incident during model evaluation, July 2026.
- Alignment faking in large language models, Anthropic, December 2024.
- A global workspace in language models, Anthropic, July 2026.
- Safety and alignment in an era of long-horizon models, OpenAI, July 2026.