Pcse00120 Page

Critics argue that these safeguards undermine the very efficiency that justifies automation. Requiring transparency and appeal processes, they claim, reintroduces delays and costs. This objection misunderstands the nature of public trust. An efficient system that routinely harms citizens is not efficient—it generates litigation, political backlash, and long-term reputational damage that far outweighs short-term processing gains. Moreover, the Dutch scandal cost taxpayers over €5 billion in reparations, dwarfing any savings from automation. Safeguards are not friction; they are insurance.

The core problem lies not with algorithms themselves but with their implementation in environments that lack due process. Consider the Dutch childcare benefits scandal (2021), where a risk-scoring algorithm falsely labelled over 26,000 families as fraudulent, leading to devastating financial ruin. Victims had no effective way to appeal the algorithm’s decisions because the system’s logic was proprietary and its errors only became visible after mass media investigation. Similarly, predictive policing tools used in Chicago and Los Angeles have been shown to perpetuate historical arrest biases, creating a feedback loop: more police presence in minority neighbourhoods generates more arrests, which the algorithm reads as evidence that those neighbourhoods require even more policing. pcse00120

These failures share a common thread: the algorithms were treated as neutral arbiters rather than as fallible tools designed by humans with implicit biases. When a human caseworker makes an error, a citizen can request a review, explain extenuating circumstances, or appeal to a supervisor. When an algorithm makes an error, there is often no comparable mechanism—just a decision score presented as objective fact. Critics argue that these safeguards undermine the very