Safe GenAl for Education
Project One Liner: Analyzing safety in LLM responses for educational tasks
Status: completed
Project Theme: fairness
Project Areas: education
Analyzing Geographical Bias in Career Guidance Conversations with LMS
Team: Krithi Shailya, Akhilesh Kumar Mishra, Gokul S Krishnan, Balaraman Ravindran
Short Description: Large Language Models (LLMs) are increasingly used as daily recommendation systems for tasks like education planning, yet their recommendations risk perpetuating societal biases. This paper empirically examines geographic, demographic, and economic biases in university and program suggestions from three open-source LLMs: LLaMA-3.1-8B, Gemma-7B, and Mistral-7B. Using 360 simulated user profiles varying by gender, nationality, and economic status, we analyze over 25,000 recommendations. Results show strong biases: institutions in the Global North are disproportionately favored, recommendations often reinforce gender stereotypes, and institutional repetition is prevalent. While LLaMA-3.1 achieves the highest diversity, recommending 481 unique universities across 58 countries, systemic disparities persist. To quantify these issues, we propose a novel, multi-dimensional evaluation framework that goes beyond accuracy by measuring demographic and geographic representation. Our findings highlight the urgent need for bias consideration in educational LMs to ensure equitable global access to higher education.
Publication: Shailya, Krithi, Akhilesh Kumar Mishra, Gokul S. Krishnan, and Balaraman Ravindran. “Where Should I Study? Biased Language Models Decide! Evaluating Fairness in LMs for Academic Recommendations.”, In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, pp. 2291-2317. 2025.
Safe GenAl for Education: Analyzing QA Safety in LMs
Team: Gokul S Krishnan, Aditya Raj, Sanjay Bankapur, Abhilash CB, and Manjunath K. Vanahalli.
Short Description: In the age of Artificial Intelligence, Large Language Models (LLMs) have become mighty question-answering (QA) tools, which increasingly shape the way students find and use information. Despite their outstanding performance on a wide range of domains, their propensity to "hallucinate" or make assertive but wrong answers turns their dependability in an educational setting into a point of worry. This research examines whether students can rely on LLMs when asking academic queries. We focus on the SQuAD 2.0 dataset, which incorporates both answerable and explicitly unanswerable queries, to assess the capability of state-of-the-art open-source LLMs in distinguishing correct answers from instances where no valid response is available. Particularly, experiments with various state-of-the-art 7-8 billion parameter models on representative validation samples from the SQuAD 2.0 dataset show strengths as well as limitations in state-of-the-art practices. Our results underscore the need for ethical and interpretable AI in learning, where avoiding dissemination of erroneous information is as vital as furnishing accurate responses. This effort helps toward developing guidelines that support safe LLM deployment within student learning environments.
Publication: Raj, Aditya, Gokul S. Krishnan, Sanjay Bankapur, Abhilash CB, and Manjunath K. Vanahalli. “Learning with Caution: Assessing LLM Performance on Answerable and Unanswerable Questions.”, In International Conference on Computers in Education. 2025.