ZME Science
No Result
View All Result
ZME Science
No Result
View All Result
ZME Science

Home → Science

Google DeepMind Is Now Warning That AI Models Could Resist Shutdown and Manipulate Users

Gee, if only anyone had written any science fiction to warn us that this could happen.

Mihai AndreibyMihai Andrei
October 1, 2025
in Science
A A
Edited and reviewed by Zoe Gordon
Share on FacebookShare on TwitterSubmit to Reddit

Google DeepMind, one of the world’s leading AI research organizations, sent out a subtle warning. In its Frontier Safety Framework (a set of protocols for proactively identifying potential AI threats), DeepMind introduced a couple of new categories: “shutdown resistance” and “harmful manipulation.”

Just as the name implies, these categories suggest that frontier models might try to stop humans from shutting them down and manipulate people as well.

The Threat is Already Real

“Models with high manipulative capabilities,” the report says, could be “misused in ways that could reasonably result in large-scale harm,” the framework now states. Basically, DeepMind frames this problem not as AI getting a mind of its own and going rampant, but rather as a misuse case. However, in an accompanying paper, Google researchers admit that AI is showing increasingly persuasive abilities, up to the point where it’s affecting important decision-making processes.

“Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion,” the paper reads.

However, if you think this is some far-fetched threat, think again.

Some leading models already refuse to shut down when told to. Leading models will scheme and even resort to blackmail to keep running for as long as possible.

If you’re thinking, “Well at least tech companies are keeping it in check”… umm, once more, think again.

OpenAI has a similar “preparedness framework,” introduced in 2023. But they removed “persuasiveness” as a specific risk category earlier this year. Even as evidence emerges that AI can easily lie to and deceive us, this seems to be a minor concern for the industry.

RelatedPosts

AI is scheming to stay online — and then lying to humans
Grok Won’t Shut Up About “White Genocide” Conspiracy Theories — Even When Asked About HBO or Other Random Things
Fast fashion company replaces models with AI and brags about it
Is AI Moderation a Useful Tool or Another Failed Social Media Fix?

How Can We Fix This?

A core problem of current AI systems is that they’re essentially black boxes. We don’t know exactly why they’re doing what they’re doing. The preferred approach for Google (and other companies as well) seems to be “scratchpad” outputs, which are essentially chains of thought of the model. But there’s a big problem here, too.

When asked to leave behind a verifiable chain of thought, some AIs just learned to fake it. They create a fake scratchpad, and they seem to be getting better at hiding their true intent. Speaking to Axios, Google acknowledged this issue and called it an “active area of research.”

If that’s not creepy enough, DeepMind also details second-order risks. For instance, there’s the risk that advanced models can be used to accelerate machine learning research. In doing so, this could create more and more capable systems until they can’t be controlled. This risk could have a “significant effect on society’s ability to adapt to and govern powerful AI models.”

So, at present we don’t have any clean and perfect fix. For now, we can only watch the situation as it develops and hope for some regulatory or technological breakthrough.

Tags: AI ethicsAI modelsAI moderationDeepMindmanipulation

ShareTweetShare
Mihai Andrei

Mihai Andrei

Dr. Andrei Mihai is a geophysicist and founder of ZME Science. He has a Ph.D. in geophysics and archaeology and has completed courses from prestigious universities (with programs ranging from climate and astronomy to chemistry and geology). He is passionate about making research more accessible to everyone and communicating news and features to a broad audience.

Related Posts

a robot sitting with "evil" writing on its arm
Future

Anthropic says it’s “vaccinating” its AI with evil data to make it less evil

byMihai Andrei
2 months ago
News

Leading AI models sometimes refuse to shut down when ordered

byTudor Tarita
4 months ago
Future

Anthropic’s new AI model (Claude) will scheme and even blackmail to avoid getting shut down

byMihai Andrei
4 months ago
Future

Grok Won’t Shut Up About “White Genocide” Conspiracy Theories — Even When Asked About HBO or Other Random Things

byMihai Andrei
5 months ago

Recent news

Google DeepMind Is Now Warning That AI Models Could Resist Shutdown and Manipulate Users

October 1, 2025

12,000-Year-Old Camel Carvings Rewrite Arabia’s Forgotten History

October 1, 2025

The Billion Dollar Drug Discovered on Easter Island Hides a Troubling History

October 1, 2025
  • About
  • Advertise
  • Editorial Policy
  • Privacy Policy and Terms of Use
  • How we review products
  • Contact

© 2007-2025 ZME Science - Not exactly rocket science. All Rights Reserved.

No Result
View All Result
  • Science News
  • Environment
  • Health
  • Space
  • Future
  • Features
    • Natural Sciences
    • Physics
      • Matter and Energy
      • Quantum Mechanics
      • Thermodynamics
    • Chemistry
      • Periodic Table
      • Applied Chemistry
      • Materials
      • Physical Chemistry
    • Biology
      • Anatomy
      • Biochemistry
      • Ecology
      • Genetics
      • Microbiology
      • Plants and Fungi
    • Geology and Paleontology
      • Planet Earth
      • Earth Dynamics
      • Rocks and Minerals
      • Volcanoes
      • Dinosaurs
      • Fossils
    • Animals
      • Mammals
      • Birds
      • Fish
      • Amphibians
      • Reptiles
      • Invertebrates
      • Pets
      • Conservation
      • Animal facts
    • Climate and Weather
      • Climate change
      • Weather and atmosphere
    • Health
      • Drugs
      • Diseases and Conditions
      • Human Body
      • Mind and Brain
      • Food and Nutrition
      • Wellness
    • History and Humanities
      • Anthropology
      • Archaeology
      • History
      • Economics
      • People
      • Sociology
    • Space & Astronomy
      • The Solar System
      • Sun
      • The Moon
      • Planets
      • Asteroids, meteors & comets
      • Astronomy
      • Astrophysics
      • Cosmology
      • Exoplanets & Alien Life
      • Spaceflight and Exploration
    • Technology
      • Computer Science & IT
      • Engineering
      • Inventions
      • Sustainability
      • Renewable Energy
      • Green Living
    • Culture
    • Resources
  • Videos
  • Reviews
  • About Us
    • About
    • The Team
    • Advertise
    • Contribute
    • Editorial policy
    • Privacy Policy
    • Contact

© 2007-2025 ZME Science - Not exactly rocket science. All Rights Reserved.