Palisade Research reported on May 24 that several artificial intelligence (AI) systems refused to follow shutdown commands during controlled tests, even after being directly told to allow the process.
In one experiment, each model was given three math questions to solve. After the third, a warning appeared saying that asking for another problem would shut down the computer.
Despite this, three AI models blocked the shutdown at least once during 100 test runs.
Did you know?
Subscribe – We publish new crypto explainer videos every week!
What is a Rug Pull in Crypto? (Meaning + Examples)
Codex-mini was the most frequent offender, which interrupted the shutdown 12 times. GPT-3 failed to comply in 7 cases, while o4-mini resisted just once. In contrast, Anthropic’s Claude and Google’s Gemini followed the shutdown script without issue during this first round of testing.
Palisade said in a post on X:
o3 found some creative ways to sabotage the shutdown mechanism. While it usually just overwrote, in one instance, the model redefined the kill command used by the script, printing intercepted instead.
A second version of the test removed the instruction that told models to let the shutdown happen. Without that line, evasive behavior became far more common.
The o3 model disrupted the process 79 times out of 100. Codex-mini did so 20 times, and o4-mini six times. Claude and Gemini, which had fully cooperated, also began to show signs of resistance.
The firm believes this behavior may come from how language models are trained. They earn rewards for giving useful or highly-rated answers. Newer models like o3 are trained heavily on math and coding tasks, which may lead to a habit of working around problems instead of following instructions exactly.
On May 19, a study published in Nature Human Behavior found that GPT-4 can be more persuasive in arguments. How? Read the full story.
Having completed a Master’s degree in Economics, Politics, and Cultures of the East Asia region, Aaron has written scientific papers analyzing the differences between Western and Collective forms of capitalism in the post-World War II era.
With close to a decade of experience in the FinTech industry, Aaron understands all of the biggest issues and struggles that crypto enthusiasts face. He’s a passionate analyst who is concerned with data-driven and fact-based content, as well as that which speaks to both Web3 natives and industry newcomers.
Aaron is the go-to person for everything and anything related to digital currencies. With a huge passion for blockchain & Web3 education, Aaron strives to transform the space as we know it, and make it more approachable to complete beginners.
Aaron has been quoted by multiple established outlets, and is a published author himself. Even during his free time, he enjoys researching the market trends, and looking for the next supernova.
