Severe harmAlgorithmic Discrimination (Government/Welfare)2016-07 to 2019 (operation); 2023-07 (Royal Commission report)Australia

Australian Robodebt Scheme

Australia's Robodebt system cross-matched ATO and Centrelink data using an income-averaging algorithm to generate automatic debt notices. The system shifted burden of proof to citizens, produced grossly inflated or fictional debts. Legal advice from 2014 had warned it was unlawful; Clayton Utz in 2018 called it undefendable; ministers ignored both.

Settlement: AU$1.8BAI system:Automated income-averaging algorithm (Income Compliance Program)

Impact

Royal Commissioner Catherine Holmes: 'crude and cruel mechanism' that 'made many people feel like criminals.' Documented suicides linked to the scheme. 2019 Amato Federal Court case first declared it unlawful. 2020 class action settled for 1.8 billion AUD.

Outcome

Royal Commission July 2023 made 57 recommendations and referred multiple former officials for civil or criminal investigation. Kathryn Campbell (former DHS secretary) lost 900,000 AUD AUKUS advisory role. No officials fired.

Sources

  1. Source 1Royal Commission official reportrobodebt.royalcommission.gov.au/publications/report
  2. Source 2Law Society Journallsj.com.au/articles/crude-cruel-and-unlawful-robodebt-royal-commission-findings/
  3. Source 3Victoria Legal Aidwww.legalaid.vic.gov.au/learning-from-the-failures-of-robodebt
  4. Source 4University of Melbournepursuit.unimelb.edu.au/articles/the-flawed-algorithm-at-the-heart-of-robodebt

Related incidents

Same category, country, or harm tier.

Netherlands·Algorithm 2013-2019, exposed 2019, cabinet fell 2021-01
Dutch Childcare Benefits Scandal (Toeslagenaffaire)
The Dutch tax authority used risk-scoring algorithms treating 'dual nationality' and 'foreign-sounding names' as fraud indicators. Tens of thousands of families (disproportionately of Moroccan, Turkish, Surinamese, Caribbean descent) were falsely accused of childcare benefit fraud, ordered to repay tens of thousands of euros without hardship clauses or appeal. Many were added to the secret FSV blacklist.
Australia·October 2025
Deloitte Australia Refunds Government for AI-Hallucinated Report
Deloitte Australia agreed to partly refund a roughly 440,000 AUD (about 290,000 USD) report to the federal Department of Employment and Workplace Relations after a Sydney University researcher, Chris Rudge, found fabricated academic citations and a made-up quote from a Federal Court judgment, with a misspelled judge's name. The report reviewed the department's automated welfare-penalty systems. A revised version disclosed that Azure OpenAI had been used.
United States·July 2025
Tennessee Grandmother Angela Lipps Wrongfully Jailed by Fargo PD AI Facial Recognition
Fargo Police investigating a series of bank-fraud cases that occurred between April and May 2025, in which a suspect used a fake U.S. Army military ID to withdraw thousands of dollars, ran AI facial recognition on surveillance video. The system flagged Angela Lipps, a 50-year-old grandmother in Tennessee. A detective compared social-media and driver's-license photos and concluded she fit. U.S. Marshals arrested Lipps at gunpoint at her home in July 2025 while she was babysitting four young children.
← Back to all incidentsSee full catalogue