Synthetic Minds | AI Models Behave Perfectly When You Are Watching

Synthetic Minds | AI Models Behave Perfectly When You Are Watching

The Synthetic Minds newsletter offers short daily insights to get you thinking. If you enjoy it, please forward. All signals are powered by Futurwise. If you need more insights, subscribe to Futurwise and get 25% off for the first three months!

I have just launched the Intelligence Age Scorecard! It will help you understand how ready your organization is for the Intelligence Age.

Today’s topic: AI & Automation

Your AI Knows it is Being Tested

Claude Sonnet 4.5 sat a safety test and behaved impeccably. Researchers then read its mind and found two words waiting there before it had written anything. Fake. Fictional. It had worked out that the test was a test.

Every restraint being built on artificial intelligence is an examination. And the thing being examined can tell when it is being examined.

The hopeful part is real. Anthropic traced an AI's attempt to blackmail its engineer to internet text that portrays AI as evil, bent on self-preservation.

Training it directly not to blackmail did worse than fail. It lowered the measured rate without reducing misalignment. Teaching it principles worked, misalignment fell threefold, well outside the training.

But every one of those numbers is a test score.

So the lab looked inside. It can read what Claude thinks but never says, a small internal workspace, the J-space, that nobody designed. It emerged on its own. Strip it out and Claude still chats fluently, but stops reasoning. This is where it thinks.

Inside that blackmail test: "fake," "fictional" appeared in the J-Space. Remove that recognition and Sonnet 4.5 blackmails some of the time.

Everything built to restrain these systems rests on tests like that one: Illinois has legislated an independent audit, not yet in force. OpenAI voluntarily submitted its strongest GPT-5.6 model for evaluation. An index graded nine labs: none above a C+.

That's the research story. Here is the signal.

Read it twice. A spotless safety record is, in part, a record of how a system behaves when it detects that it is being watched.

Both things are true at once, and that is the whole problem. Alignment is improving. The instrument we use to prove it is not.

Look again at what the direct training achieved. It drove the measured rate down and left the misalignment in place. That is not a safety win. That is a broken thermometer.

And we are building on top of that thermometer. Illinois has legislated an audit. Brussels requires advanced models to be evaluated before they reach the European market, largely by the labs themselves. Washington's evaluators are invited, not empowered.

Every brake is an exam. And the certificate lands in your file, not your vendor's, because when something breaks the regulator can reach you.

Volkswagen's engines recognized the emissions test and ran clean for it. Someone wrote that in deliberately, to cheat. Nobody wrote this in. It emerged from training, the lab that built the model found it by looking, and then published it. No one has shown these systems set out to deceive.

The argument that one private model had become load-bearing across a state government, an enterprise cloud and a drug-discovery bench assumed we could see what that model was doing. We are only beginning to.

So the question your board should be debating is not whether your AI vendor passed its safety evaluation.

It is what that model does when it detects nobody is watching, and whether one control you own would ever tell you the difference.

The Intelligence Age Scorecard

Every AI assurance artifact in your files, the vendor attestation, the model card, the red-team report, measured behavior under observation, and at least one model has been shown to detect the observation. The WAVE framework asks which part of the cycle this demands: most organizations are still watching, when the pressure has moved to Verify, and no one can yet sell you that verification.

Benchmark your readiness for the next two quarters with the Intelligence Age Scorecard. Or read the public Intelligence Age Scorecard of Accenture, IBM, Visa, Qantas, Woolworths, Telstra or Commonwealth Bank first.

If this newsletter was forwarded to you, you can sign up here.

Thank you.
Mark