{"id":148073,"date":"2022-12-14T15:09:33","date_gmt":"2022-12-14T14:09:33","guid":{"rendered":"https:\/\/www.erasmusmagazine.nl\/en\/?p=148073"},"modified":"2025-12-19T11:26:34","modified_gmt":"2025-12-19T10:26:34","slug":"anti-cheating-software-could-be-racist-says-human-rights-institute","status":"publish","type":"post","link":"https:\/\/www.erasmusmagazine.nl\/en\/2022\/12\/14\/anti-cheating-software-could-be-racist-says-human-rights-institute\/","title":{"rendered":"Anti-cheating software could be racist, says human rights institute"},"content":{"rendered":"

Robin Pocornie, a student at VU Amsterdam, was sometimes unable to access the online environment of the anti-cheating program Proctorio because her face was not recognised. She suspects that is because of the colour of her skin.<\/p>\n

\"\"

\n\t\n\n<\/svg>\n<\/span>\n\tImage by:<\/span>\n\n\t\t\t\n\t\t\t\t\t\t\tEsther Dijkstra<\/a>\t\t\t\t\t<\/span>\n\t<\/p>\n<\/figcaption><\/figure>

The software was frequently used during the coronavirus pandemic to prevent fraud. Invigilators could then observe students taking an exam at home. The software incorporates facial detection and apparently Pocornie\u2019s face could sometimes not be found. This meant that she was occasionally ejected from the exam with error messages like \u2018face not found\u2019 and \u2018room too dark\u2019.<\/p>\n

Enquiry<\/h2>
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>

Centre: Robin Pocornie. Left and right: Hans de Zwart and Naomi Appelman of the Racism and Technology Center, which is advising her.<\/span><\/p>

\n\t\n\n<\/svg>\n<\/span>\n\tImage by:<\/span>\n\n\t\t\t\n\t\t\t\t\t\t\tracismandtechnology.center<\/a>\t\t\t\t\t<\/span>\n\t<\/p>\n<\/figcaption><\/figure><\/div>

She filed a complaint at her university, which denied that there was any discrimination. The university also asked the supplier about it, but the supplier said that there was nothing wrong with the software. According to Proctorio, that emerged from an external investigation.<\/p>\n

VU Amsterdam says an unstable internet connection is a better explanation of the problems than the colour of this student\u2019s skin. After all, others had no similar complaints relating to their skin colour and there were in fact more students who had problems logging in.<\/p>\n

An interim decision<\/a> shows that the Netherlands Institute for Human Rights does not go along with that. It is, after all, beyond dispute that the student had problems logging in. Moreover, scientific research shows that facial recognition software generally works less well with dark-skinned people.<\/p>\n

Not public <\/h2>

The university\u2019s counterarguments do not weight up against that, in the opinion of the institute. And the external investigation carried out on Proctorio\u2019s behalf? It has not yet been made public. In addition, the university has not given out any information about the average login time of other students.<\/p>\n

VU Amsterdam now has ten weeks to refute the suspicion of discrimination, otherwise the ruling will go against the university. The decisions are not binding, however. And the institute cannot impose a fine or award damages.<\/p>\n

\u201cI\u2019m very happy that the law is on my side\u201d, Pocornie says in a statement to the press. She is curious to see the reaction of other higher education institutions that have also used this software.<\/p>\n

Important <\/h2>

The human rights institute is satisfied too. \u201cAlthough this is only an interim decision, it is an important moment in the history of our decision-making\u201d, says institute chair Jacobine Geel in a news item on the website. \u201cSomeone has succeeded in saying, \u2018Hey, this algorithm is doing something odd. I suspect the algorithm is discriminating.\u2019\u201d<\/p>\n\n\t\n\t

\n\t\t\t\t\t

De redactie<\/h2>\n\t\t\n\t\t
\n\n\t\t\t\n\t\t\t