Are we really adapting? How AI can contribute to tracking global progress

Efforts have been undertaken to adapt our environments to the inevitable effects of climate change. Or, at least, that is what we assume and, most of all, hope. So far, we've been lacking insight in the extent to which adaptation progress is being made, especially on global scale. To make sure we know what is (and is not) happening and design appropriate new policies accordingly, the need for a large-scale framework to consistently track climate change adaptation - which I'll refer to as 'adaptation' from now on - is urgent. But, how can we do this while the evidence base of climate policies is so voluminous, unstructured, multifaceted, and expanding continuously, while expert annotators are expensive and scarce, and time is ticking?

Countries have created various adaptation policies and have been reporting on their actions in many different documents. This evidence is, however, often hidden in large volumes of fluffy texts, jointly covering other topics like climate change mitigation or development policy. Although the evidence we need is already there, it is crucial that we first increase structure and quality and, by doing so, increase accessibility of adaptation-specific information to (among others) researchers and policy-makers, allowing for monitoring and evaluation of adaptation practices across the globe.

The potential of Artificial Intelligence (AI)

Nowadays, the solution to such large-scale text analysis problems seems not too far away: AI can just do it for us, right? Well, this is partially true: feed ChatGPT a document, ask it to conclude about adaptation progress, and it will likely give you a sense-making answer. Problem solved, you'd say.

Well, taking any model's output as the ground truth may lead to biases analyses, hidden inaccuracies, and a lack of transparency. As the quality of evidence used to track adaptation is highly important, we need to make sure we find the best way to benefit from AI in our analyses. Should we use the general knowledge of advanced large language models (LLMs)? Should we train smaller models to context-specific tasks? Or are there other AI-driven methods that have more potential?

Narrowing down adaptation evidence

In my PhD research, I try to get an answer to these questions. To do so, we first broke down the framework into separate tasks, with the eventual goal of bringing structure to the landscape of adaptation policies. At the core of this is the data of Climate Policy Radar (CPR), which is a non-profit building open, credible databases and AI powered tools to support informed climate, nature, and development action. CPR's efforts in collecting documentation and building an open database of climate laws and policies have been of great help in tackling the first challenge: gathering the data we need.

To first narrow down the evidence to what is actually relevant, we evaluated a wide range of state-of-the-art AI approaches (among which LLMs and various models fine-tuned on our own data) to identify blocks of texts that contain any bits that involve adaptation.

This led to a paper, presented at ACL 2025 during the 2nd Workshop of NLP meets Climate Change. In this paper, we revealed our fine-tuned ClimateBERT model (available through HuggingFace) as the top candidate for the task, showing superiority over the hyped, computationally costly LLMs and traditional query-based approaches. Applying this model to the global dataset led to a novel database of adaptation-relevant text blocks.

Next: mapping adaptation policies

Although the first step in increasing accessibility of adaptation data has been achieved, we're obviously not there yet. Together with my valuable team of trained data annotators, who labelled and categorised raw data, we've created a sampled dataset of adaptation goals, instruments, and outputs identified in multiple thousands of policy text blocks.

Using this data, we will identify the most suitable AI-driven method for "mapping" adaptation policies: revealing what are the specific goals, instruments, and outputs, their sub-categories, and - crucially - the connections across these elements. These elements will help us finally grasp the state of climate change adaptation per country (over time), help assess global and national patterns, and the connections will reveal potential gaps in, for example, ambitious goals lacking concrete instruments.

In the study, we compare two main methods. The first one involves a language model, which we fine-tuned to directly retrieve and classify policy elements from input texts by generating structured outputs. We compare its outputs to a more complex, multi-agent AI framework that combines the general capabilities of large language models (LLMs) and smaller, task-specific models, fine-tuned on our own labelled data. To get to a final choice, we first use quantitative evaluation: here, we simply calculate the overlapping elements across the human, "ground truth" annotations and those extracted by the AI models. In addition, we conduct a qualitative, blind review in which a sample of human "ground truth" annotations is compared to the AI annotations on a few criteria.

With this, we intend to reveal interesting insights in the use of "AI annotators" in the climate policy context and, following, conduct a global, empirical analysis using our novel, structured evidence base of climate change adaptation policies.

Jetske Bonenkamp is a PhD candidate in the Public Administration and Policy Group and the Artificial Intelligence Group at Wageningen University and Research, located in the Netherlands. She is researching how state-of-the-art Artificial Intelligence (AI) methods - Natural Language Processing (NLP) in particular - may contribute to large-scale adaptation tracking, as part of NWO-VIDI project "High ambitions, (s)low implementation? The politics of tracking adaptation to climate change".After completing a BSc in Industrial Design Engineering, Jetske earned a Master's degree in Interaction Technology at the University of Twente, specialising in AI/NLP.