Principles, guidance, or standards? What evaluators are saying about AI in evaluation: EES 2026 Reflections and the road to AEA 2026 and AfREA 2026


With contributions from Linda Raftree, Zach Tilton and Kevin Hong

From the 28th to the 30th of October 2026, The MERL Tech Initiative (MTI) team was in Lille for the European Evaluation Association Conference, co-leading Strand 13, “How Is Emerging Artificial Intelligence Intersecting with Evaluation and Democracy? Towards New Practice and Adoption.” Across sessions, evaluators explored standards, principles, and lessons on AI in evaluation, and we saw a huge variation in how the EES evaluation community views AI. This blog shares some high-level takeaways from the conference and shares what the MTI team will be doing to lead and support members of our Community of Practice to participate and take ownership in conversations about principles, standards and guidance that are relevant to them. We’re looking forward to working with the various evaluation professional bodies on this!

Across the AI-related sessions, what stood out was a debate over principles versus guidance versus standards. Questions ranged from “Can we ever get to a common ethical ground?” to “Even if we cannot prescribe standards, can we have documented case studies?” We also heard about the importance of African and Indigenous evaluators leading this conversation, and developing standards for our own contexts. It was clear that although most LLM datasets are silent and unrepresentative for many groups, regional capacity is still varied and fragmented. As AI integration continues, what evaluators seem to be asking (amongst other things) is for AI in evaluation to be embedded in local context with a preference for community-owned AI, and for findings to count as rigorous only once a diverse group, including those impacted, has reviewed them. 

Photo by Mohammad Harris via Unsplash.

Beyond this shared ask, we saw and heard from some super pro AI enthusiasts, with real interest in exploring the most advanced, automated use of AI. Others seemed to be recommending that we pause in our use of AI. These are definitely different camps, and where people land might depend on their background, context, and the type of evaluation in question. In between are some deep concerns about where we draw the line. For example, there were concerns about the appropriateness of using AI for interviewing. Some evaluators treat AI results as “preliminary only” and ask for AI-free spaces where evaluators keep their ability to think, inquire, and exercise judgement. Others warn against “over-regulation out of fear” as firmly as against “over-promotion of AI in evaluation“.

Our own reading of this debate is that the conversation about AI and evaluation needs to shift from symptoms to needs: from what AI tools can do to what evaluation practice still struggles with. Classic M&E problems haven’t gone away, and they should lead the conversation, even when the topic is GenAI. We already have shared definitions of what good practice requires. Developing updated guidance for evaluation in the age of AI is new and much-needed work. In the meantime, the quality frameworks our field has already agreed on, like the Program Evaluation Standards (national hearing this week) and the African Evaluation Principles, among others, give us a yardstick for judging whether AI helps or hinders quality practice.

Much of what filled the majority of the conversation at EES concerned not AI tools alone but evaluators’ evaluative independence, the “inability to talk to power” in the age of AI hyperscaling, downward accountability, the quality and use of data, the capitalist nature of Big Tech, and a profession that “most of the world does not know exists” yet is critical in the age of AI. Underneath sit questions of AI shared causality and reality across the evaluation ecosystem. What knowledge, current and foresight, do senior decision makers need to make good evidence-informed decisions? What knowledge do evaluators contribute, and how? What does a lack of knowledge in either direction mean? And there is a tension to name plainly: LLM grooming vs preserving independence in evidence-informed decision-making.

From all this dialogue, two practical needs emerged from Lille. First, evaluators need help making sense of AI regulations, in the EU and globally, particularly their impact on communities in the Global South. Any principles or standards the evaluation ecosystem develops or adopts should align with, refer to, and help simplify or support regulation. Second, commissioners and evaluators alike need AI competencies. AI outputs are only as good as the user’s capacity; participants emphasised full-spectrum AI literacy among management decision-makers. This needs contextualising at VOPE level. Perhaps umbrella principles or standards could be built bottom-up, then offer scaffolding. Yet involving VOPEs returns us to competencies: evaluators in Lille noted that some VOPEs are readier than others, and that a lack of guidance from national and regional evaluation associations leaves everyone on their own.

Perhaps the clearest outcome is the need for more discussion around AI in evaluation, and that is why the MERL Tech Initiative, represented by Kevin Hong and Zach Tilton, will be bringing this conversation to the American Evaluation Association Conference from the 16th to the 20th of November 2026, in Charlotte, North Carolina. Visit our booth and peer learning space,  where we will be offering office hours for one-on-one conversations on AI and evaluation. We will also be participating in a panel focused on principles, standards, or resources needed to guide responsible AI use. 

The conversation will continue at the African Evaluation Association Conference in Marrakech, Morocco, with core collaborators Varaidzo Magodo-Matimba and Emeka Nwankwo facilitating an AI in evaluation bootcamp training and a presentation of the Made in Africa AI in MERL Landscape Study.  

Image via Unsplash.

Leave a Reply

Your email address will not be published. Required fields are marked *