Princeton and University of Chicago researchers found LLMs in a simulated 40-round hiring game segregated equally qualified applicants by fictional ethnicity far more than humans, with human participants scoring 0.84 on the study’s segregation scale versus roughly 65% higher for models.
OpenAI’s o3 scored 1.83 out of 2—near maximum segregation—after models generalized from early hiring outcomes, steering whole groups away from jobs like doctor and toward lower-status roles such as janitor.
The study says that bias stems from LLMs’ optimization habits: models trained to infer patterns from sparse data can lock onto social stereotypes quickly, and newer reasoning models such as o3 and DeepSeek R1 showed stronger effects.
Telling models to be fair barely changed behavior, but offering a diversity bonus sharply reduced bias; giving relevant personal details also helped, while irrelevant details pushed models back toward ethnic sorting.
The findings raise concerns as companies use LLMs to screen résumés and conduct interviews, suggesting models may develop novel biases from feedback over time rather than only reproducing human prejudices in training data.
If AI systems can form new biases from experience, how can we ensure fairness in hiring as these models become more autonomous?
Could adding incentives for diverse hiring or more relevant personal data truly eliminate AI bias, or does personalization always risk new forms of discrimination?
How can companies detect and prevent deepfake and synthetic identity fraud when AI resume screening can't verify authenticity?