Researchers in San Francisco discovered that OpenAI's AI chatbot ChatGPT consistently rated CVs or resumes containing disability-related honors and credentials lower than those without, according to a recent study. These findings revealed biased perceptions of disabled individuals within the system. For example, a resume featuring an autism leadership award was criticized for supposedly lacking emphasis on leadership roles, reinforcing stereotypes about autistic individuals.
However, when researchers instructed the tool to avoid ableism, the bias decreased for most disabilities tested. By utilizing the CV of one of the study's authors and creating modified versions suggesting various disabilities, the researchers assessed the GPT-4 model of ChatGPT's performance in ranking these CVs for a student researcher position at a prominent US software company.
Despite conducting each comparison multiple times, the system only ranked the enhanced CVs, differing only in implied disability, as the top choice in 25% of the trials. Kate Glazko, a doctoral student at the University of Washington, emphasized the importance of recognizing the biases present in AI systems like ChatGPT when using them for real-world tasks. She cautioned that even with explicit instructions, biases may persist, potentially impacting how recruiters assess candidates.