How to Participate in the AEA/MTI Virtual Hackathon 2.0

Hackathons aren’t just for software developers. At their core, they’re structured spaces for learning in public — trying something, sharing it, and building on what others create. For evaluators navigating a fast-moving AI landscape, that kind of collective experimentation matters. In that sense, the hackathon is a small-scale version of what Donald Campbell called an “experimenting society” — a community committed to trying things, learning from what happens, and using that knowledge to do better. That’s what we’re after here.

That’s exactly why AEA and MTI are partnering for a second hackathon — and why this one looks a little different from the first.

What Changed from 1.0

The first AEA/MTI Hackathon was in-person at AEA24 in Portland. It was energizing — but overwhelming for some, inaccessible to many, and more competitive than collaborative. Hackathon 2.0 fixes that. It’s virtual, bite-sized, and built around sharing over winning. Submissions go into a live gallery so participants can learn from each other, and prizes are distributed by lottery rather than ranking.

Three Paths, One Hackathon

To make participation accessible across interests and experience levels, the hackathon is organized into three paths and multiple activities. You pick the path and activities that fit where you are right now.

Critique brings your evaluative thinking to AI. Critique a claim about AI in evaluation, revise AEA competencies for AI-enabled practice, or write a case study on a success or failure of using AI in evaluation. 

Create is the traditional hackathon spirit — build something new through a prompt engineering challenge, a tiny chatbot build, or a more ambitious vibe-coded tool. 

Collab is for those who prefer to iterate: start from a shared Claude Artifact template, customize it, and share what you changed with the community.

How It Works

Participation is designed to fit a busy schedule. Start by visiting the hackathon launch page and picking your path. From there, dig into one of the path activities — most take somewhere between 10 and 30 minutes, though you’re welcome to go deeper if you’re in a flow state. When you’re done, submit your entry through a short form. Each submission asks for four things: a one-line description of the need or claim you were working on, evidence of your work, a quick ethics check to confirm no PII is included, and a 140-character reflection on what you did or learned. After submitting, your entry appears in the live gallery alongside everyone else’s — which is where a lot of the learning actually happens. You can submit up to three entries total, and each one earns you a lottery entry for prizes, which include comped AEA conference tickets, membership fees, training offerings from MTI, and multiple $100 dollar gift cards.

Frequently Asked Questions

Do I have to be a member of AEA or the NLP CoP?

No membership required — the hackathon is open to anyone.

How long does it take to participate?

Activities are designed to take 10–30 minutes.

How long will the hackathon be open?

The hackathon runs from April 12–25th. You can participate anytime during that window.

Do I need any subscriptions?

No subscriptions needed. The hackathon is platform agnostic, so you can use whatever AI tools you already have access to.

What if I need support during the process?

There’s a dedicated support channel in the NLP Community of Practice Slack Workspace. You can join the workspace and get help from the community throughout the hackathon window.

What about intellectual property of my submission?

Submissions follow a CC 4.0 license. You own your work — others can use and build on it as long as they reference you.

How can I access the hackathon launch page?

Access will be granted to participants the week of April 12. We will share the link on our blog! Until then, you can complete this form to be notified when the hackathon starts.

Get Involved

If you’d like a deeper orientation before diving in, we held a webinar walking through the hackathon in full — including a live demonstration of each path. It covers everything above and leaves time for Q&A.

The hackathon opens April 12th. Bring a colleague — this kind of learning is better with company. Reach out anytime with questions: Zach Tilton at zach@merltech.org or Zachary Grays at zgrays@eval.org.