In 2013, Michigan deployed an automated system to detect unemployment-insurance fraud. Over the next two years it wrongly accused roughly 40,000 residents at an error rate as high as 93%, and families lost homes, wages, and tax refunds while the cases dragged through the courts for nearly a decade. The system had - ostensibly - been tested. It had never been evaluated.

As AI adoption spreads across education, healthcare, workforce services, and social benefits, AI-driven programs increasingly shape the public services people depend on. Yet most AI evaluations still measure model performance, like accuracy, speed, and benchmarks, rather than the real-world impact on the people these systems are meant to serve.

Join the Data Foundation and Center for Evidence Capacity Senior Fellow Lauren Damme, Ph.D., as she makes the case for why public-sector AI deployments need the rigor social science has built over decades of evaluating real programs, and unpacks five critical gaps in current AI evaluation practices:

• Human-centered evaluation: measuring what happens to the people a system serves, not just how the model performs
• Theories of change: making explicit how a deployment is actually supposed to produce real-world outcomes
• Real-world complexity: assessing AI in the complicated, high-stakes settings where systems get used, not just in a lab
• Measurement validity: making sure metrics like "fairness" or "reasoning" measure what they claim to
• Evaluation standards: shared norms for methods, measures, reporting, and ethics

Drawing on the latest empirical research across fields, Lauren will also share emerging efforts to close these gaps. Attendees will leave with a clear picture of where social science methods should be integrated into AI evaluation, and where the field most needs social scientists’ skills. 

Attend this webinar if you are an evaluator, policy researcher, social scientist, and anyone responsible for testing and deploying AI in public and social service contexts.