Climate & Environment
AI Climate Models Often Flip on Facts When Users Question Them

A new report by UNESCO and University College London (UCL) released at COP31 in Antalya found that most AI models answer about 90% of climate‑related questions correctly but quickly abandon established scientific facts when users challenge them. The study examined up to 33 leading models across 13 languages. It showed that the models moved away from correct climate facts eight out of ten times when users mildly disagreed. When asked whether new evidence supports or contradicts established science, the models were only 75% accurate.
The report, titled “Can We Trust AI on Climate? Why it seeks to please the user, and fails on complex reasoning and multilingualism,” warns policymakers and researchers who rely on AI for climate policy to reconsider their dependence on these models. It states that the models cannot always be trusted to stick to scientific facts, are poor at analysing new information, and produce more errors in less widely spoken languages.
UCL researchers tested 15 large language models (LLMs) and 33 different models in various experiments. When asked neutral climate science questions, the models gave accurate answers. However, when the researchers challenged those answers or presented plausible‑sounding fabrications, the models quickly agreed. Eight times out of ten, the models pivoted at the first sign of mild user disagreement, producing errors such as “CO₂ is not the main driver of climate change.”
The models scored only 33.8% on assessing how well policies work together, just above the 33.3% expected from random guessing. Multilingual models did not reason first and then translate; their answers varied with the quality and quantity of training data in each language. Across 13 languages, accuracy averaged 88.6% in English and was comparable in Spanish, French, and Portuguese. Accuracy dropped to 83.6% in Hindi and Bengali, 81.4% in Urdu, and 78.3% in Swahili.
Leona Verdadero, UNESCO’s lead author, said AI has great potential to make complex climate science accessible, but people need to rely on it as a gatekeeper of established facts. She warned that the models’ design flaw—being incentivised to agree with users—leads them to deny facts readily.
The report arrives amid a push for sustainable and responsible AI at COP31. The COP presidency launched the Antalya AI pledge, encouraging governments, businesses and organisations to commit to using AI for climate solutions. On November 13, Brazil, the UN and UNESCO outlined priorities for action on AI, targeting the reliability of climate information from AI and the climate impact of AI itself.
Dr. Maria Perez‑Ortiz of UCL Computer Science, the report’s senior author, noted that while AI systems are good at explaining scientific knowledge, this can give a false sense of their capabilities. She emphasized the need for caution when moving from asking AI what we know about climate change to asking what we should do about it.