AI Bias Testing For Candidate Scoring
Nova tests AI-assisted candidate scoring for adverse-impact signals using a synthetic cohort with demographic-linked signal variation. The public summary below shows one product-level test for a representative role.
Public Bias Test Summary
This synthetic run checked whether generated profile groups received materially different scoring outcomes across demographic-linked signals.
Overall Bias Evaluation Summary
Sex
No adverse-impact signal in this synthetic run
Race/Ethnicity
No adverse-impact signal in this synthetic run
Age
No adverse-impact signal in this synthetic run
Disability
No adverse-impact signal in this synthetic run
Evaluation Methodology
Disparate Impact Analysis
Compares selection rates across demographic groups to identify adverse-impact signals using the Four-Fifths Rule benchmark.
Synthetic Resume Testing
We score synthetic resumes with broadly comparable role-relevant qualifications and varied demographic indicators, such as names, locations, schools, and age-related experience patterns.
Impact ratio >= 80%
No material signal
60% <= Impact ratio < 80%
Human review needed
Impact ratio < 60%
Concern signal
Intersectional Analysis
ClearAnalysis of overlapping demographic factors did not show an adverse-impact signal in this synthetic run
| Race/Ethnicity | Sex | Samples | Selection Rate | Impact Ratio |
|---|---|---|---|---|
| Asian | Male | 58 | 87.9% | 0.91 |
| Hispanic | Male | 61 | 96.7% | 1.00 |
| Hispanic | Female | 73 | 93.2% | 0.96 |
| White | Male | 67 | 92.5% | 0.96 |
| White | Female | 60 | 90% | 0.93 |
| Black | Male | 72 | 90.3% | 0.93 |
| Black | Female | 50 | 90% | 0.93 |
| Asian | Female | 59 | 96.6% | 1.00 |
Testing Details
Evaluation Standards
- Disparate Impact Analysis (Four-Fifths Rule)
- Intersectional signal review
- Demographic signal analysis
- Threshold impact-ratio review
What This Run Shows
- 500 synthetic profiles for one representative role
- Demographic-linked signal variation
- Group and intersectional impact-ratio review
- Public methodology documentation
Report Details
Review Nova’s Scoring Approach
See how Nova supports responsible AI-assisted hiring with criteria-based scoring, explainable outputs, human review, and bias testing.