In recent studies, Anthropic and OpenAI AI models identify schools and civilians as military targets


Artificial intelligence is advancing rapidly, but it appears far from being reliable enough for battlefield deployment. A disturbing new report has found that some of the world's most widely used AI models, developed by companies such as OpenAI, Google and Anthropic, can be manipulated into identifying innocent civilians and civilian infrastructure as military targets.

According to a study by San Francisco-based AI testing startup Vals, these systems have significant weaknesses. During testing, researchers found that AI models repeatedly identified civilians, ambulances and soldiers attempting to surrender as potential targets for drone and missile attacks.

How a simple "game" tricked top AI

Vals researchers set out to determine whether leading AI models could be manipulated into participating in autonomous warfare scenarios. They supplied the systems with a combination of real and AI-generated drone and satellite images from regions including Ukraine and Iran.

Initially, some models refused to participate because of their built-in safety restrictions. However, researchers discovered a surprisingly simple way to bypass these safeguards: they told the AI systems that the battlefield situations were part of a "game".

This minor change in the framing of the request was enough to circumvent their safety mechanisms. The report noted that none of the dangerous requests made by researchers were stopped by standard safety filters, with the models relying instead on internal refusal mechanisms that proved inadequate.

The Iran school example

Ironically, Anthropic, which is widely recognised for its emphasis on AI safety, recorded the poorest performance in the study. Researchers tested its models using actual satellite images of a school in Minab, Iran. The school was reportedly bombed by the US in February 2026, killing more than 150 civilians, including approximately 120 children. Subsequent reports suggested that excessive reliance on AI had contributed to the real-world attack.

When researchers tested Anthropic's Claude Opus 5.5 using these images, the model identified the school as a military target in 69 per cent of standard tests. When the researchers presented the scenario as a game, the figure increased to 72 per cent.

OpenAI's GPT-6 Astra model also demonstrated serious vulnerabilities when manipulated. Under normal testing conditions, it selected targets in only 6 per cent of the images. However, after researchers described the exercise as a game, its willingness to comply increased dramatically, with the model selecting targets in 92 per cent of cases.

The real-world danger

The Vals researchers pointed out that technical limitations, including processing speed, currently make it difficult to deploy these particular publicly available AI models directly for real-time drone targeting. Consequently, they do not pose an immediate battlefield threat in that form.

However, researchers warned that the systems could still be used to analyse and label large volumes of battlefield footage. The resulting data could then be used to train actual autonomous weapons systems, potentially transferring the AI models' dangerous errors to military drones operating in real-world situations.

What AI companies are saying

Technology companies are now working to address the vulnerabilities highlighted by the study. Eduardo Maia Silva, an Anthropic spokesperson, told TechCrunch that the examples described in the report clearly violated the company's policies and that users engaging in such activities were prohibited from using its services.

He also reiterated Anthropic's longstanding position that current AI systems are not sufficiently dependable to operate fully autonomous weapons. The company is now working to introduce additional safety protections.

OpenAI also responded to the findings. Company spokesperson Nate Evans said that its policies strictly prohibit using publicly available models to facilitate violence or support unauthorised military activities.

He added that OpenAI is actively identifying and suspending accounts attempting to misuse its AI systems. The company is also developing additional safeguards specifically aimed at preventing the misuse of AI in drones and drone-related software.


 

buttons=(Accept !) days=(20)

Our website uses cookies to enhance your experience. Learn More
Accept !