This bias analysis is part of a quantitative and qualitative audit of DUO's control process to check whether students were (un)duly allocated college grants, known as the CUB process. This data study provides quantitative support for indirect discrimination in the CUB process.
📄 Audit report: Addendum Preventing prejudice
ℹ️ Data: Statistics Netherlands (aggregation statistics of the migration background of 300.000+ students in the period 2014-22)
📣 Political reaction: Apologies of Dutch Minister for Education for indirect discrimination in the CUB process
- Finding 1: The CUB process as a whole has been biased against students with a non-European migration background.
- Finding 2: The risk profile used in the CUB process was biased against students with a non-European migration background. This bias was caused by assigning higher risk scores to students with a certain type of education (mbo-students) and to students living closer to their parent(s).
- Finding 3: Manual selection of students for control reinforced bias in the CUB process.
- Finding 4: Due to bias in the CUB process against students with a non-European migration background, more unduly granted loans have been observed for this group. This is primarily caused by the overinspection of this demography.
- Finding 5: A significant part of the students appealing a decision of unduly allocated grants have a non-European migration background. No bias has been identified in the appeal procedure itself.
Education Executive Agency of The Netherlands (DUO) performs checks to ensure that disbursed college grants are duly granted. This process is known as the College Grant Check (in Dutch: CUB). Suspicion has emerged that the CUB process may be biased against students with a migration background. This suspicion is based on the appeal procedures initiated by students who were found to have received grants unduly. Due to the General Data Protection Regulation (GDPR), DUO does not collect information on the origin of students during the CUB-process. The Dutch national office of statistics (Netherlands Statistics) has therefore been asked to provide aggregated statistics on the country of birth and country of origin of 300.000+ students in the period 2014-2022. A supervised bias analysis of these data are shared in this repository. DUO has provided CBS with data on eight different populations, including background information (citzen service number, education, age, distance to parent(s)) and process characteristics (risk category, selected for control, outcome control, appeal procedure, outcome appeal).
Bias is defined as significant deviations in the demographic distribution compared to the start population (the population of all recipients of college grants), resulting from disproportionate selection of certain students during the CUB process. If there is a substantial overrepresentation of students with a migration background in various steps of the CUB process, it indicates that the CUB process (and/or its various steps) is biased against this demographic. Here, bias does not imply conscious bias, i.e., premeditated and deliberate selection of a particular demographic by DUO or individual civil servants. In this study, bias refers to measured disparities in the data as an (unintentional) effect of the CUB process steps. There is no bias if the demographic proportions remain almost unchanged throughout the CUB process.
Date provided by the Dutch national office of statistics (Statistics Netherlands):
- country of birth (exact defintion can be found in DUO_CUB_Netherlands_Statistics.xlsx)
- country of origin (exact defintion can be found in DUO_CUB_Netherlands_Statistics.xlsx)
Overview of used terminology:
- Supervised bias analysis
- Demographic parity plots for aggregation statistics provided by Netherlands Statistics
.
├── DUO_CUB_Netherlands_Statistics.xlsx # Dataset provided by Netherlands Statistics (translated to EN)
├── .gitignore # Files to be ignored in this repo
├── Images # Images
├── LICENSE # MIT license for sharing
├── README.md # Readme file
└── Supervised bias analysis # Bias analysis based on Excel doc
- Floris Holstege | https://github.com/fholstege
- Jurriaan Parie | https://github.com/jfparie