Stop DUO-style fraud detection, advises state commission

The government must make a greater effort to combat racism and discrimination, according to a special state commission. It should, for example, stop using ‘data-driven profiling’ such as that previously employed by student finance agency DUO.

Taking action only after the fact is not enough, the State Commission against Discrimination and Racism writes in its final report. “Tackle discrimination and racism proactively.”

The commission was established four years ago in the aftermath of the childcare benefits scandal. Its task was to map discrimination and racism in society and within government.

One of the recommendations in the report is to stop using data-driven profiling in the delivery of public services and government tasks. If citizens are filtered on the basis of postcode, educational level, income or other characteristics, prohibited distinctions can easily creep in.

“Recent scandals have shown how deeply discrimination and racism can become embedded in policy, algorithms and implementation practices”, the report states. The commission also cites student finance agency DUO as an example.

Students living away from home

When investigating fraud involving the basic grant for students living away from home, students with a migration background were disproportionately targeted, according to an investigation published three years ago by the Higher Education Press Agency, platform Investico, NOSop3 and newspaper Trouw.

The findings led to an apology from the government, reimbursement of fines and compensation payments for the students who had been discriminated against. The state commission would rather prevent such problems than correct them afterwards.

That is not always easy: visible discrimination is often addressed, but ‘more subtle, deeply embedded forms persist’, the commission argues. Sometimes discrimination is difficult to recognise, for example in a fraud-detection algorithm ‘that in practice disproportionately selects certain groups’.

Profiling

Last month, the commission had already published a report on data-driven profiling. “In its checks on grants for students living away from home, DUO did not sufficiently investigate which alternatives to the algorithm were available”, it stated.

Even then, the commission’s conclusion was that the government should stop using data-driven profiling. Instead, it could carry out random checks without prior filtering or, alternatively, inspect everyone. “These methods reduce the risk of discrimination, are easier to explain and contribute to equal treatment and public trust.” The commission also highlights a financial advantage: the government runs a lower risk of incurring high remediation costs.

And if the government nevertheless wants to assess who may be suspicious in the future? It should first demonstrate that the use of profiling is necessary and proportionate, the commission argues. Moreover, it should be tested to ensure that it does not have a discriminatory effect.

DUO is indeed working on a new system for detecting students who misuse the maintenance grant for students living away from home. Education Minister Rianne Letschert has promised that the system will undergo an independent review before it is introduced.