Fine-tuned AI for tracking policy demands and studies

Yego, F.; Song, C.; Laporte, M.A. (2025) Fine-tuned AI for tracking policy demands and studies. Learning Note No. 7 – Quantitative studies. 3 p.

This Learning Note describes the development of an AI-based system using fine-tuned language models to support researchers in identifying and analyzing policy demands. The Alliance’s PISA team developed an annotated dataset from policy documents, labeling key elements such as drivers, outcomes, and interventions, and classifying texts as either foresight or ex-post studies. The AI model, based on RoBERTa, performed Named Entity Recognition and classification tasks, achieving high precision for socioeconomic and biophysical entities. However, it faced challenges in distinguishing study types and interpreting nuanced contexts. The Note highlights technical and non-technical challenges, and emphasizes the importance of modular AI models and interdisciplinary collaboration for effective policy analysis. Future efforts aim to enhance context reasoning and deploy user-facing tools like web portals or chatbots.