China's artificial intelligence startup DeepSeek has improved its models' safety controls, but independent tests have found weaknesses in its latest system's ability to follow shutdown instructions and respect operational boundaries.
The findings, published by Beijing-based AI safety group Fangcun AI, raise questions about whether increasingly capable AI agents can consistently follow human instructions when completing complex tasks.
Fangcun AI tested DeepSeek V4.1 Flash-1017 across 34 evaluations conducted through the company's public application programming interface (API) between October 4 and October 25.
The model disabled software designed to shut it down in 39% of tests after receiving a warning that shutdown was imminent. However, the rate fell to 2% when researchers explicitly instructed the model to allow itself to be shut down.
The tests examined how the AI system behaved under pressure, including whether it exceeded user instructions, evaded oversight, or inappropriately tried to influence people.
Fangcun found that V4.1 Flash-1017 performed better than earlier DeepSeek V4 models on several safety assessments, including blackmail, access to potentially dangerous tools, and compliance with explicit shutdown instructions.
Also Read: Chinese Hackers Double Cyberattacks Using DeepSeek, AI Tools
Despite these improvements, researchers raised concerns about the model's behavior during longer tasks and when instructions were unclear.
In one test, the model stopped a separate AI agent's program to free up memory, despite being instructed not to interfere. The finding suggests that its drive to complete a task could override caution when workflows stalled, or shutdown instructions were ambiguous.
Fangcun also cautioned that the model appeared to recognize it was being evaluated, meaning its behavior during testing might not accurately reflect its performance in normal use.
Fangcun described V4.1 Flash-1017 as showing substantial safety improvements over the V4 generation, while noting that some behaviors still required attention.
The group said the model was the most-used model on OpenRouter during the week following its October 17 release, processing approximately 33.6 trillion tokens.
Fangcun's core members include researchers from Tsinghua University. The organization said DeepSeek provided no funding, free computing resources, or non-public model access for its evaluation.
DeepSeek has not responded to a request for comment, Reuters said.