FairFrame: An Evaluative Framework That Detects and Helps Eliminate Bias in LLM/NLP Models

Geoffrey Levy ’26

Faculty Mentor: Dr. Yaghoobian

What is the broader impact of your summer research? That is, why should the world care about your work?

AI is becoming increasingly prevalent in everyday life, but these models are trained on data that contains bias. If these tools are to be fair and representative, we must be able to detect and ultimately mitigate this bias. We worked on taking two programs used for detecting bias in LLM/NLP models (Checklist and Hugging Face’s Evaluate) and integrated them to create a program that allows for enhanced bias detection using numerous different metric tests.

What has summer research taught you about yourself and the field that you researched and/or studied?

This summer has allowed me to become a lot more effective with identifying gaps in my knowledge. There would be points where I had a hard time understanding a concept, which prompted me to become better at troubleshooting on my own as opposed to falling back on the structured outline of a traditional course. I’ve learned that in the field of computer science, particularly with machine learning/artificial intelligence, research can be very interdisciplinary. With my research area, not only is it a computer science/mathematical problem, but a sociological one.

What advice would you give to students considering a future research opportunity similar to the one you participated in?

Recognize that this is an evolving process. Your first research opportunity can be a jumping-off point for you or others which can be continued either throughout future semesters or during future summer sessions. Keeping good notes can aid in the continuation of this research.