Evaluation Function
Permanent URI for this collectionhttps://hdl.handle.net/10568/108606
Browse
Recent Submissions
Item Management Response Systems to Evaluations: Benchmarking Review(Report, 2025-02) Ahmedou AbdallahiItem Evaluations of CGIAR Science Groups: Synthesis on Quality of Science(Report, 2025-01) Jill Lenné; SG Evaluation TeamsEvaluating the quality of science, research, and innovation is an integral part of achieving and measuring the process and progress of the SDGs (EvalSDGs Insight #18). The CGIAR’s Guidelines on Evaluating QoS (also Spanish) elevated QoS as a cross-cutting criterion within the six OECD/DAC criteria for framing evaluative judgments and as a stand-alone evaluation criterion with four dimensions (design, inputs, processes, and outputs). The QOS elements of credibility and legitimacy are embedded in the QoS criterion. The value and utility of the criterion were demonstrated through a summary of the Evaluative Evidence from the CGIAR Portfolio on Quality of Science (2020-2023 reviews and evaluations of CGIAR Research Programs (CRPs) and Platforms. The learning informed the design of the Science Group Evaluations (Terms of Reference) and data collection methods to answer two overarching questions: ● To what extent do the SGs' management processes ensure the QoS, including credibility, legitimacy, relevance to next-stage users, and potential effectiveness, of the research and operations? ● In what ways are the research outputs by the SGs of high quality and influential? In 2024, evidence on the QoS inquiry from the evaluations of the Science Groups (SGs): Genetic Innovation (GI), Resilient Agri-food Systems (RAFS) and System Transformation (ST) was synthesized across the four QoS dimensions and complemented by 11 case studies and in-depth analyses an online survey, key informant interviews, and Altmetric Attention Scores (AAS) to assess influence. Findings were analyzed against the OECD/DAC criteria of relevance, coherence, efficiency, and effectiveness despite initial challenges in applying QoS guidelines. Intended for evaluators, policymakers, and researchers involved in CGIAR and related initiatives, the CGIAR Science Group Evaluations: Brief on Quality of Science asses QoS through the four key dimensions earlier mentioned. The report includes valuable lessons from each Science Group and outlines recommended actions for improving QoS within the 2025–2030 CGIAR Portfolio, emphasizing stronger alignment, stakeholder engagement, and interdisciplinary collaboration. To dive deeper into the findings and recommendations: • Download the full report: Access detailed insights, lessons learned, and actionable steps to drive high-quality research and innovation aligned with the SDGs. • Visit the Science Group evaluations portal: Explore comprehensive evaluations and reports on Quality of Science. • Explore the QoS Resource Hub: Access tools, guidelines, and resources to support research excellence and impactful innovation.Item CGIAR Science Group Evaluations: Brief on Partnerships(Brief, 2025-01) Independent Advisory and Evaluation ServiceAligned to the CGIAR 2030 Research and Innovation Strategy, CGIAR has been expanding its global network of partners and changing the way it works with them to drive the transformation of food, land, and water systems. Guided by the 2022 CGIAR Engagement Framework for Partnerships and Advocacy, the portfolios of CGIAR’s Science Groups-Resilient Agrifood Systems (RAFS), Genetic Innovation (GI), and Systems Transformation (ST)-aimed to implement a more structured approach to partnerships, focusing on co-developing research, enhancing capacity sharing for development, and strengthening collaboration with the private sector. This brief highlights key evaluative insights and lessons on partnerships from the GI, RAFS and ST Science Group (SG) evaluations. Conducted by the IAES, in alignment with the evaluation Terms of Reference (ToR), the three SG evaluations were carried out under the 2022‒24 Multi-Year Evaluation Workplan (2021; re-confirmed 2024). From the three SG evaluations and their Synthesis-including the online survey and several case studies- recommendations on partnerships from the 2021 Synthesis of Learning from a Decade of CGIAR Research Programs were considered. Formal recommendations and learnings, aim to strengthen CGIAR’s institutional learning, accountability, and effective and efficient implementation of the 2024 Partnership Framework (update from 2022) under the CGIAR 2025-30 Portfolio. Stay tuned for more insights and recommendations on the SG Evaluations Portal.Item Systems Transformation Science Group: Evaluation Report(Report, 2024-11) K. Belbase; C. Fabricius; J. Howard; J. Zake; L.C. Corsetti; D. CekovaThe CGIAR 2030 Research and Innovation Strategy set the stage for ensuring that research provides solutions for development. The strategy was delivered through 33 initiatives grouped within three interlinked action areas: Systems Transformation (ST), Resilient Agri-food Systems, and Genetic Innovation. The ST Science Group (SG) contributes to food, land, and water across CGIAR’s five Impact Areas, supporting policy and decision-makers at multiple governance levels with timely policy-relevant insights. This independent evaluation of the ST SG was carried out under the framework of SG evaluations in response to the Terms of Reference. The evaluation combines summative and formative aspects to support learning and accountability among key users and stakeholders. This evaluation report summarizes findings, conclusions, recommendations and lessons learned from implementing the 12 initiatives under the ST SG from 2022-24. The evaluation also considers the SG's objectives, activities, and management processes with a forward-looking perspective for the next science program portfolio 2025-30. This report evaluates ST initiatives from January 2022 to February 2024, assessing their relevance, quality of science, coherence, efficiency, and effectiveness. The report identifies strengths, such as robust scientific standards, collaborative efforts, and progress in outputs despite resource challenges, and areas needing improvement, including stakeholder engagement, financial management, and strategic alignment. The evaluation provides actionable recommendations to enhance transformative research, interdisciplinary collaboration, partnerships, and the next portfolio's design. By focusing on climate resilience, food systems, and country-level strategies, the report aims to optimize CGIAR's contributions to sustainable development and global impact. Explore more resources and insights on the Science Group Evaluations Portal.Item Resilient Agrifood Systems Science Group: Evaluation Report(Report, 2024-11) N. Palmieri; B. Bennett; K. Fakoya; C. Meisner; M.M. MolinariThe CGIAR 2030 Research and Innovation Strategy sets priorities to deliver solutions for development through 33 initiatives across three interlinked action areas: Systems Transformation, Resilient Agrifood Systems (RAFS), and Genetic Innovation. The RAFS Science Group (SG) delivers on research priorities aimed at transforming agri-food systems to enable the most vulnerable to access affordable, sufficient, safe, and healthy diets through 15 initiatives. This RAFS SG evaluation report is one of three SG evaluation reports, adopting a forward-looking perspective for the next portfolio 2025-30. In response to the evaluation Terms of Reference, the report evaluates the RAFS SG Portfolio from January 2022 to February 2024, focusing on its effectiveness, relevance, coherence, efficiency, and quality of science. The evaluation aims to provide an independent assessment of performance and identify lessons learned to guide future design. Key findings include strong alignment with global agricultural priorities, as well as challenges in cross-center collaboration and stakeholder engagement. Recommendations emphasize enhancing national research involvement, strengthening country-level coordination, improving operational efficiency, and refining strategic planning for the 2025-30 portfolio. The report also advocates for better integration of gender, social inclusion, and climate change considerations in future research programs. Explore more resources and insights on the Science Group Evaluations Portal.Item Genetic Innovation Science Group: Evaluation Report(Report, 2024-11) R. Paz Ybarnegaray; B.I.G. Haussmann; D.H. Meyer; G. J. Bryan; S. E. AsareThe CGIAR 2030 Research and Innovation Strategy sets the stage for research that provides solutions for development. Priorities set out in the strategy were delivered through 33 initiatives grouped within three interlinked Action Areas: Systems Transformation, Resilient Agri-food Systems, and Genetic Innovation (GI). This report evaluates the GI Science Group (SG), with evaluative scope covering the GI SG portfolio implementation from January 2022 to February 2024, excluding the Genebank Platform, which was evaluated in 2023. The evaluation criteria, in response to the evaluation Terms of Reference, include relevance, coherence, effectiveness, efficiency, and quality of science. The evaluation findings and recommendations inform portfolio design and the development of new GI and science program proposals. The evaluation also contributes to strategic decision-making by the CGIAR System Council about the 2025-27 Multi-Year Evaluation Plan, informing portfolio design and the development of new GI and science program proposals. By employing a theory-based approach with mixed methods, the report sought to understand how well GI SG activities aligned with national and regional needs, including those of farmers and end-users. The evaluation also aimed to identify challenges in breeding programs, such as integration with genebanks and other SGs, while highlighting successes in market intelligence, partnerships, and gender inclusion. Ultimately, the goal was to provide actionable recommendations for improving strategies and practices to ensure sustained impact in crop breeding and food systems. Explore more resources and insights on the Science Group Evaluations Portal.Item Terms of Reference: Review of CGIAR’s Management Response System to Independent Evaluations(Report, 2024-10) CGIAR Independent Advisory and Evaluation ServiceItem CGIAR Genebank Platform: Evaluation Report(Report, 2024-03) S. Humphrey; D. Coombs; H. Pritchard; JL. Pham; S. Sellitti; S. AnandThe purpose of the 2023 evaluation of the Genebank Platform was to support the institutional learning of Crop Trust and CGIAR and provide evidence regarding the efficiency and effectiveness of the Genebank Platform (from 2017-21). The evaluation exercise aimed to also identify good practices and lessons for the Genebanks Initiative and Genetic Innovation Science Group and CGIAR at large. Towards evaluation credibility, data collection followed a mixed methods design, leveraging both qualitative and quantitative data from primary and secondary sources to answer evaluation questions, understand operating environments, and track contextual and programmatic assumptions. Eight key evaluation questions were framed along the five evaluation criteria: relevance, effectiveness, efficiency, coherence and added-value, and sustainability and learning for impact (see CGIAR-wide Evaluation Policy). The data collection incorporated semi-structured interviews, in-depth document and literature reviews, field visit(s) to Genebanks, and strategically identified gatherings of Platform stakeholders for face-to-face interactions. Accordingly, three platform module studies, two deep dive studies, an online survey, and an exploratory social network analysis were developed as the core source of evidence for developing evaluation conclusions and recommendations. All Genebank Platform stakeholders were welcome and encouraged to engage in the evaluation process and participate in interviews. Please contact IAES-Evaluation@cgiar.org for related inquiries.Item Terms of Reference: CGIAR Science Group Evaluations(Internal Document, 2024-01) CGIAR Independent Advisory and Evaluation ServiceThe evaluation function of the Independent Advisory and Evaluation Services (IAES), is executing the 2024 evaluations of the three Science Groups (SGs)—Systems Transformation (ST), Resilient Agrifood Systems (RAFS), and Genetic Innovation (GI). The evaluation Terms of Reference (TORs), which are endorsed by the Strategic Impact, Monitoring, and Evaluation Committee (SIMEC) and build on the Concept Note, are presented to the CGIAR System Council (SC19) as a pre-read. The SG evaluations align with CGIAR's Evaluation Framework and Policy, and stem from the CGIAR 2030 Research and Innovation Strategy. Between 2022-23, the 2030 CGIAR Research Portfolio comprised of 33 Initiatives across three Action Areas (i.e., Science Groups). The objectives of the SG-level evaluations are two-fold: firstly, to furnish real-time feedback and recommendations, towards institutional learning and adaptation of the CGIAR 25 Portfolio; and secondly, to facilitate accountability for, and learning from, the initial two years of 2030 portfolio implementation. This evaluation would also aid SIMEC and IAES to identify evaluative needs for the 2025-27 multi-year evaluation plan (MYEP). Aligned to the nascent status of the CGIAR Portfolio, the 2024 formative real-time evaluations would strategically focus on relevance, coherence, effectiveness, efficiency, and quality of science (QoS) evaluation criteria. Among others, key lines of inquiry encompass comparative advantage, compliance with relevant CGIAR frameworks and policies, and integration of cross-cutting themes (e.g., gender and climate change, partnerships, capacity development). The evaluations’ timeline would span from March to October 2024, involving learning from data collection and analysis for 'real-time' engagement with CGIAR management. The System Council endorsement is set for October 2024, and would feature SG-level Management Response (MR), developed by CGIAR management, and approved by the CGIAR Board. The participatory evaluation process aligns with CGIAR's commitment to continuous improvement, learning, and transparency. Stakeholder engagement is pivotal to ensure that diverse perspectives contribute to the collective insights that will guide CGIAR's evolution. For further inquiries or engagement in the evaluation process, stakeholders are encouraged to reach out to IAES-Evaluation@cgiar.org.Item Social Network Analysis: Evaluation of the CGIAR Genebank Platform(Internal Document, 2024-01) Anand, Shweta; Humphrey, Sarah; Sellitti, Stefania; Coombs, David; Pritchard, Hugh W.; Pham, Jean-LouisThe Social Network Analysis (SNA) was part of the 2023 evaluation of the Genebank Platform. The SNA aimed to identify and delineate the main stakeholders and study the relationships between themselves and other stakeholders as they interacted on CGIAR’s Genebank Platform. The analysis focused on comprehending the flows of information between these different partners and users. The use of SNA in the evaluation complemented and supplemented the three key criteria of relevance, effectiveness and coherence, and aided results obtained for the other three Platform modules, namely, policy, conservation, and use. Key takeaways from the SNA included the need for CGIAR to continue its efforts to strengthen its relationship with the NARS in each country.Item CGIAR GENDER (Generating Evidence and New Directions for Equitable Results) Platform: Evaluation Brief(Report, 2023-10) CGIAR Independent Advisory and Evaluation ServiceItem Evaluation of CGIAR Genebank Platform: Results from Online Survey(Report, 2023-10) CGIAR Independent Advisory and Evaluation ServiceItem Evaluation of CGIAR Excellence in Breeding Platform (EiB)(Report, 2022-07) CGIAR Independent Advisory and Evaluation ServiceItem Elaboración y uso de los análisis de evaluabilidad en el CGIAR(Report, 2022-10) CGIAR Independent Advisory and Evaluation ServiceItem Aplicación del Marco de la QoR4D del CGIAR a las evaluaciones de procesos y desempeño(Report, 2023-01) CGIAR Independent Advisory and Evaluation ServiceItem Applying the CGIAR Quality of Research for Development Framework to Process and Performance Evaluations(Report, 2023-01) CGIAR Independent Advisory and Evaluation ServiceItem CGIAR GENDER Platform Evaluation: Inception Report(Report, 2023-02) CGIAR Independent Advisory and Evaluation ServiceItem Advisory Report: Study on the Performance Results Management System (PRMS) Project Management Approaches and Fit-for- Purpose Information Products(Report, 2023-06) CGIAR Independent Advisory and Evaluation ServiceItem Terms of Reference: Evaluability Assessments of the Regional Integrated Initiatives, CGIAR(Internal Document, 2023-09) CGIAR Independent Advisory and Evaluation ServiceThe CGIAR Independent Advisory and Evaluation Services (IAES) 2022‒24 multi-year workplan provides for the evaluability assessments of Regional Integrated Initiatives (RIIs) in 2023. These ToRs respond to the mandate of the CGIAR IAES Evaluation Function to execute the evaluability study; and sets out the purpose, objectives and a process to follow, in accordance with the CGIAR Evaluability Assessment Guidelines. The primary purpose of the Evaluability Assessment is to ensure evaluability and preparedness for an evaluation, aligned to different placing in the project cycle. The goal of the assessment is to support the RIIs in preparing for successful evaluations, which can involve assessing the adequacy of the ToC, establishing the interests of the stakeholders, and determining whether available data would make it possible to formulate a judgment given the initiative timeframe and resources. Guided by IAES budget consideration, the RAFS Action Area director, and subsequently RII coordinator (appointed July 2023), the following criteria were applied: (1) status of the Initiative in the project cycle and funding levels; (2) explicit interest and/or availability from the RIIs; and (3) geographical representation, with potential for deeper learning at a regional level. The selected RIIs are: Fragility to Resilience in Central and West Asia and North Africa (CWANA); Asian Mega-Deltas (AMD); Transforming Agrifood Systems In South Asia (TAFFSA); West and Central African Food Systems Transformations (WCA). The EA approach will follow utilization-focused and collaborative, as well as focus on accountability and learning. This approach will be mixed methods of data collection and analysis: (1) documents and data reviews: a desk review of secondary data sources: documents and related resources; and (2) primary data collection: key informant interviews and focus group discussions. Please contact IAES-Evaluation@cgiar.org for related inquiries.Item CGIAR GENDER Platform: Evaluation Report(Report, 2023-09) Podems, D.; Stathers, T.; Forsythe, L.; Mbevi, L.; Highet, C.; Sweitzer, E.; Maharjan, S.The summative and formative evaluation of the GENDER (Generating Evidence and New Directions for Equitable Results) Platform assessed its progress from January 2020 to October 2022, to document lessons and best practices, to also provide forward-looking recommendations for the newly transitioned CGIAR GENDER Impact Platform. The evaluation was conducted between October 2022 – June 2023 by the external team, which was led by an evaluation expert and comprised of three subject matter experts to specifically address Evidence, Methods, and Alliances modules of the Platform. This evaluation employed theory-based, utilization-focused, participatory, and feminist approaches aligned with CGIAR’s Evaluation Framework and Policies. The Platform evaluation readiness was determined by an evaluability assessment jointly carried out by the GENDER platform team, with facilitation and quality assurance under IAES, following the CGIAR evaluability assessment guidelines. The presentation of approaches and methodology of the evaluation and its novelties was shared with the wider audiences at the 2023 gLOCAL event: How Evaluations Contribute to Addressing The Greater Societal Issues: Evaluation of the CGIAR GENDER Platform Additional detail on methodology is provided in the Annex. Seven key evaluation questions were framed along the five evaluation criteria: relevance, effectiveness, efficiency, coherence, and sustainability. Quality of Science evaluation criteria was integrated into the specific evaluation sub-questions of the Evidence module study, to pretest the evaluation guidelines CGIAR on evaluating QoR4D in process and performance evaluations. The evaluation used three kinds of triangulation: investigator triangulation, data triangulation, and methodological triangulation. Mixed methods were employed, including: online survey (119 respondents- see survey blog), key informant interviews (84 participants), and desk reviews on Platform indicator data, annual reports, module financial data, and other secondary documents. Executive summaries of three module reports are includes in the Annex, complementing the Platform-wide evidence and conclusions. Responding to the key stakeholder groups, the evaluation made 11 recommendations for: the GENDER Platform and CGIAR management. Complementing lessons from other platform evaluations of Excellence in Breeding Platform and Big Data in Agriculture, lessons learnt from this evaluation of the GENDER platform that would apply to new CGIAR impact platforms. Please read the evaluation report with stand-alone annexes, and Management Response to the evaluation. Stay tuned for other knowledge management products.