Keeping the Human in the loop: Balancing AI capabilities with Human Judgement in Evaluation
Guest post by Prabin Nanicha Shrestha
The next meeting of the AI, Asia-Pacific and MERL Learning Group is happening on November 24, from 12:00 to 1:00 PM IST.
Artificial intelligence is creating new opportunities across the evaluation cycle, from evaluation design and data analysis to synthesis and reporting, while raising important questions about human judgment, contextual understanding, evidence quality, ethics, and accountability. This session will explore how evaluators can meaningfully integrate AI into their practice while keeping humans at the centre of evaluation. Through practical examples and a moderated discussion, speakers will reflect on where AI can add value, where human oversight remains essential, and how evaluators can balance innovation and efficiency with methodological rigour and responsible practice, particularly in the evolving evaluation landscape across Asia and the Pacific.
The session will be organized in collaboration with EvalYouth Asia (EYA) to strengthen regional connectivity and broaden engagement across the evaluation community, particularly among young and emerging evaluators in Asia and the Pacific. This collaboration will support wider outreach and participation, while creating opportunities for emerging evaluation professionals to contribute to conversations on the responsible and meaningful integration of AI into evaluation practice.
Get to know the speakers
Arshee Rizvi is an AI and data transformation leader, evaluator, and development professional working at the intersection of artificial intelligence, public policy, and inclusive development. She holds an M.Tech from IIT Bombay and is an alumna of the International Program for Development Evaluation Training (IPDET) at the University of Bern, Switzerland. Her work spans AI/ML, data-driven governance, climate resilience, and monitoring, evaluation, and learning (MEL). She has conceptualised and deployed an offline, multilingual AI tool for elected women representatives that received appreciation and support from the Government of India. Arshee serves on the Boards of Directors of the International Organisation for Co-operation in Evaluation (IOCE) and the Asia Pacific Evaluation Association (APEA), is a Core Member of the Evaluation Community of India (ECOI), and Co-Leader of EvalYouth India. She is passionate about advancing ethical, inclusive, and accessible AI to strengthen evidence-informed decision-making and development outcomes.
Will Allen is an independent systems scientist, evaluator, facilitator, and writer with more than 30 years’ experience across sustainable development, natural resource management, climate adaptation, biodiversity, freshwater, and collaborative governance. He supports people working where research, policy, communities, practice, and Indigenous and Western knowledge need to meet. A growing part of his thinking and writing concerns how generative AI can be used well in evaluation, facilitation and knowledge work, particularly where human judgement, transparent criteria and ethical reasoning matter. He has an extensive background working at the intersection of technology, communication and collaboration, from early work with videotext and internet-based decision-support systems to the continuing development of Learning for Sustainability. Will’s current work explores where AI can strengthen professional practice, and where its use needs greater scrutiny.