Danaë Metaxa (University of Pennsylvania): “The Future of Algorithm Auditing is Sociotechnical”

Presentation Abstract:

Algorithm audits are powerful tools for studying black-box systems without direct knowledge of those systems’ inner workings. While they have been effectively deployed to identify harms and biases in algorithmic content, algorithm audits’ narrow focus on technical components stop short of considering users themselves as integral and dynamic parts of the system, to be audited alongside its algorithmic components.

After an overview of the state of the art in algorithm auditing, this talk will introduce sociotechnical auditing: evaluating algorithmic systems at the sociotechnical level, focusing on the interplay between algorithms and users as each impacts the other over time. I will also present Intervenr, a platform we developed to conduct browser-based, longitudinal sociotechnical audits, and a case study in which we deployed Intervenr to investigate the central claim of online targeted advertising systems: that targeted ads perform better with users. Finally, I will conclude by discussing some of my group’s current work, expanding the auditing method to novices and youth, and developing post-auditing tools for collective action.

Speaker Bio:

Dr. Danaë Metaxa is a Raj and Neera Singh Term Assistant Professor of Computer and Information Science at the University of Pennsylvania, with a secondary appointment in the Annenberg School for Communication. Dr. Metaxa specializes in issues of bias and representation in algorithmic systems and content, across high-stakes social settings including politics, employment, and education, and their work has been awarded at venues including CHI and CSCW. Dr. Metaxa’s most recent work has focused on online ad targeting, including its effectiveness and its impact on marginalized populations. Prior to UPenn, Dr. Metaxa received their PhD from Stanford University in Computer Science.