Severe harmAlgorithmic Discrimination (Government/Welfare)2016-07 to 2019 (operation); 2023-07 (Royal Commission report)Australia

Australian Robodebt Scheme

Australia's Robodebt system cross-matched ATO and Centrelink data using an income-averaging algorithm to generate automatic debt notices. The system shifted burden of proof to citizens, produced grossly inflated or fictional debts. Legal advice from 2014 had warned it was unlawful; Clayton Utz in 2018 called it undefendable; ministers ignored both.

Settlement: AU$1.8BAI system:Automated income-averaging algorithm (Income Compliance Program)

Impact

Royal Commissioner Catherine Holmes: 'crude and cruel mechanism' that 'made many people feel like criminals.' Documented suicides linked to the scheme. 2019 Amato Federal Court case first declared it unlawful. 2020 class action settled for 1.8 billion AUD.

Outcome

Royal Commission July 2023 made 57 recommendations and referred multiple former officials for civil or criminal investigation. Kathryn Campbell (former DHS secretary) lost 900,000 AUD AUKUS advisory role. No officials fired.

Sources

  1. Source 1Royal Commission official reportrobodebt.royalcommission.gov.au/publications/report
  2. Source 2Law Society Journallsj.com.au/articles/crude-cruel-and-unlawful-robodebt-royal-commission-findings/
  3. Source 3Victoria Legal Aidwww.legalaid.vic.gov.au/learning-from-the-failures-of-robodebt
  4. Source 4University of Melbournepursuit.unimelb.edu.au/articles/the-flawed-algorithm-at-the-heart-of-robodebt

Related incidents

Same category, country, or harm tier.

Netherlands·Algorithm 2013-2019, exposed 2019, cabinet fell 2021-01
Dutch Childcare Benefits Scandal (Toeslagenaffaire)
The Dutch tax authority used risk-scoring algorithms treating 'dual nationality' and 'foreign-sounding names' as fraud indicators. Tens of thousands of families (disproportionately of Moroccan, Turkish, Surinamese, Caribbean descent) were falsely accused of childcare benefit fraud, ordered to repay tens of thousands of euros without hardship clauses or appeal. Many were added to the secret FSV blacklist.
United States·July 2025
Tennessee Grandmother Angela Lipps Wrongfully Jailed by Fargo PD AI Facial Recognition
Fargo Police investigating a series of bank-fraud cases that occurred between April and May 2025, in which a suspect used a fake U.S. Army military ID to withdraw thousands of dollars, ran AI facial recognition on surveillance video. The system flagged Angela Lipps, a 50-year-old grandmother in Tennessee. A detective compared social-media and driver's-license photos and concluded she fit. U.S. Marshals arrested Lipps at gunpoint at her home in July 2025 while she was babysitting four young children.
South Korea·August 2024, ongoing
South Korea Deepfake Sex Abuse Crisis
In August 2024, journalists and activists exposed hundreds of Telegram channels where male students at Korean schools and universities uploaded AI-generated sexually explicit images of female classmates, teachers, family members, and soldiers. One channel had 220,000+ subscribers. Police recorded 297 deepfake cases in first 7 months of 2024 (vs. 180 in all 2023); 74% of suspects were 10-19 years old.
← Back to all incidentsSee full catalogue