AI Accountability for For-Purpose Organisations in Australia
The Federal Court has approved Australia’s largest class action settlement for Robodebt victims. It is not just a story about government failure. It is a story about what happens when algorithmic systems make decisions about vulnerable people and nobody is accountable for the result. For for-purpose organisations, the accountability gap exists both in their own AI use and in the algorithmic systems their clients are subject to every day. This article examines what that means and what accountability looks like in practice.
Observations
When Nobody Is Accountable, Nobody Is Safe
This week, the Federal Court approved Australia’s largest class action settlement in history. The government will pay $475 million in compensation to more than 125,000 people affected by the Robodebt scheme, on top of $1.8 billion already paid and nearly two million dollars in forgiven debts. The total cost to government exceeds $2.4 billion.
Robodebt was not a rogue AI. It was an automated data-matching system that compared welfare recipients’ reported income against averaged tax office data and issued debt notices without human oversight. It ran for six years. It affected 470,000 Australians. The Royal Commission found it contributed to at least three deaths by suicide, and that senior public servants knew it was likely unlawful from the start.
Nobody signed off on what it was doing to people. That is not a technical failure. It is an accountability failure. And it is not over.
In this article
What Robodebt proves about algorithmic accountability and who pays when it fails
The two kinds of invisible AI already operating in the for-purpose sector
Why the for-purpose worker is the only human in the loop for clients subject to systems they cannot see
What accountability looks like in practice, and a free resource to help build it
Photo: Serge Le Strat / Unsplash
AI Accountability for For-Purpose Organisations in Australia
Two Kinds of Invisible AI
Two kinds of invisible AI.
Across the for-purpose sector, the conversation about artificial intelligence is mostly about the AI organisations are choosing to use: tools for drafting grant applications, summarising case notes, generating reports, supporting staff under pressure. That conversation is necessary and overdue.
But there is a second conversation that is barely happening at all.
The communities that for-purpose organisations serve are already subject to algorithmic decision-making they did not choose and cannot see. The system that calculated whether a welfare recipient owed a debt. The algorithm that determines how much support a person with disability receives under the NDIS. The risk assessment tool that influences whether a family stays together. The data-matching model that shapes housing allocation. These systems are making consequential decisions about people’s lives every day, and in most cases the people affected have no visibility of how those decisions were reached, no language to contest them, and no pathway to a named human who takes responsibility.
Robodebt was the most visible example of what happens when that accountability gap is left open. It will not be the last.
470,000
Australians issued with automated debt notices under Robodebt. The system ran for six years before it was stopped. No human signed off on individual decisions. Three deaths by suicide were linked to the scheme.
AI Accountability for For-Purpose Organisations in Australia
The Federal Court’s approval of the Robodebt class action settlement proves what happens when algorithmic systems make decisions about vulnerable people without human accountability. For for-purpose organisations in Australia, the accountability gap exists both in their own AI use and in the automated systems their clients face every day. Building visible accountability structures is not a compliance requirement. It is the lesson Robodebt has already written into Australian legal history.
The Research
The systems your clients cannot see.
Photo: Zhenyu Luo / Unsplash
Research published in the Australian Journal of Social Issues examined how the NDIS uses algorithmic decision support systems to develop personalised support plans. It found these systems raise significant concerns about fairness, accountability, transparency, and participation in decision-making. For participants who lack the social capital or documentation to advocate for their needs, undue weight given to algorithmic inputs can deprive them of entitlements they are legally owed.
A 2026 study published by the ACM examined how people with communication disabilities navigate digitised welfare services and what they anticipate from AI-automated futures. Participants described systems they could not access, could not understand, and could not contest without trusted human intermediaries. They used words like humiliating, overwhelming, and designed to deny. They did not want AI removed from welfare. They wanted AI that was transparent, accountable, and retained a pathway to a named human being.
A parallel study of frontline homelessness workers in Toronto found that algorithmic systems used to prioritise housing and support services were built on data collected under conditions of mistrust, repeated questioning, and resource scarcity. The data did not reflect the complexity of people’s lives. The algorithms built on that data could not either. The human workers knew this. The systems did not account for it.
In welfare automation, there are typically two classes of AI: interactional systems that mediate access and gather evidence, and decision-making systems that determine eligibility and trigger case outcomes. In practice, both operate together. The person affected by the decision rarely knows either exists.
The Only Human in the Room
The only human in the room.
For-purpose organisations occupy a specific and largely unacknowledged position in this landscape. Their workers are often the only humans present at the intersection of two accountability gaps: the gap in their own organisation’s AI use, and the gap in the algorithmic systems their clients are subject to. This is the lived reality of autonomy and judgement in the sector, and it sits squarely within community and equity concerns that AI governance has barely begun to address.
Research published by the University of Cambridge and the Alan Turing Institute examined social work practitioners’ experiences of AI across the UK sector. Every one of the practitioners interviewed expressed unanimous aversion to algorithmic decision systems. Not because they were technophobic. Because they understood something the systems did not: their work requires contextual human judgement that cannot be flattened into a risk score. As one participant put it, you cannot capture in data the feel of when you go into a house.
But the same practitioners described being stretched, under-resourced, and operating under conditions that make sustained critical engagement with any system genuinely difficult to maintain. The burden of being the human in the loop, for both their own AI use and their clients’ invisible algorithmic context, was falling on workers who already had very little capacity to carry it.
This sector will be the first responder when algorithmic harm reaches the community level. It needs structures for accountability, not just intentions.
100%
Share of social work practitioners interviewed in a 2024 Cambridge and Turing Institute study who expressed aversion to algorithmic decision systems in their work. Not because AI has no role. Because they understood what the systems could not account for.
Source: Wassall, K. et al. (2024). Reimagining AI in Social Work. Cambridge / Turing Institute
The Confidence Problem
The confidence problem, compounded.
Research published in April 2026 by the American Psychological Association found that passive acceptance of AI output, accepting what the system produces without questioning or modifying it, is associated with lower confidence in independent reasoning. The issue was not AI use itself. It was the degree of engagement. Workers who actively pushed back on AI output retained confidence in their own judgement. Those who did not reported reduced confidence and a diminished sense of ownership over their own thinking.
Consider those findings against the conditions in which community sector workers operate. High caseloads. Resource scarcity. Decisions made under time pressure with incomplete information. These are precisely the conditions the APA research found most associated with passive AI acceptance.
The for-purpose worker sitting across from a client who has been denied a welfare payment by an algorithm they cannot see is being asked to exercise confident, independent human judgement on behalf of that client. At the same time, they may be using generative AI in their own work under conditions that the research suggests quietly erode that same capacity.
The accountability gap is not only in the systems their clients face. It is in the conditions their own work creates. Combined with automation bias, the result is a compounding risk that no policy statement currently addresses.
The issue was not AI use itself, but the degree of passive acceptance. Participants who used AI but maintained oversight and active judgement tended to feel more confident in their own reasoning. Those who accepted outputs with minimal modification reported less.
Source: Baldeo, S. (2026). Technology, Mind, and Behavior, APA
What Accountability Looks Like
What accountability looks like in the room.
Photo: National Cancer Institute / Unsplash
The Robodebt Royal Commission’s most damning finding was not that an algorithm made errors. It was that the humans responsible gave little thought to the impact on the people affected. Accountability was assumed to exist because a system was in place. Nobody checked whether it actually did.
The for-purpose sector does not have to repeat that pattern. But avoiding it requires more than good intentions. It requires structures that make human involvement visible, documented, and reviewable at the point where AI contributes to consequential work.
Not For Humans has developed the AI Authorship Statement, a free resource designed specifically for for-purpose organisations. It is not a compliance certificate. It is a professional declaration that a named human was in the loop, made judgements, and takes responsibility for the output.
It asks the human to declare their role before a document is submitted or used. It gradates scrutiny by the stakes of the work. It creates a reviewer loop so that accountability is documented rather than assumed.
It is the practice that Robodebt proves cannot be left to chance. We would genuinely like to know how it lands in your organisation.
Email to request your copy →For Leaders
What this asks of organisational leaders.
Australia’s Privacy Act will impose new obligations on automated decision-making from December 2026. Government is moving, slowly, toward greater transparency requirements for algorithmic systems in public services. That policy layer is necessary and welcome.
It will not be sufficient on its own.
Policy creates obligations. Structures create practice. The distance between a transparency requirement written into legislation and a worker in a community organisation who genuinely understands how AI contributed to a document they just submitted is exactly the distance that Robodebt exposed. This is the work of governance and accountability in practice, not on paper. It is what Not For Humans calls practice over paper. And the community sector will face this as a cascading demand long before government guidance reaches operational level.
For organisational leaders, the questions worth sitting with are not about compliance. They are about practice.
When AI contributes to a significant piece of work in your organisation, is there a named human who takes responsibility for what it produced? Not in principle. In writing.
When a client your organisation supports is affected by an algorithmic decision they do not understand, does your team have the language and the capacity to help them navigate it?
When something goes wrong, will there be a record of who was in the loop, what they checked, and what they decided?
These are not auditing questions. They are the questions Robodebt makes unavoidable.
Photo: Romain Dancre / Unsplash
In Closing
The price of an empty chair.
The $2.4 billion cost of Robodebt is not primarily a financial figure. It is the documented price of an accountability gap left open for six years. It is what happens when automated systems make consequential decisions about vulnerable people and nobody asks who is responsible for what the system is doing, or what it is doing to real lives.
For-purpose organisations are not government agencies running welfare compliance programmes. But they are operating in the same landscape, using AI in their own work while their clients are subject to AI in systems they cannot see or contest.
The question is not whether to use AI. It is whether the humans using it, and the humans affected by it, have visible accountability structures around them.
Robodebt is not a cautionary tale about what AI might do one day. It is a settled legal finding about what it already did. And the court has now put a number on the cost of leaving the chair empty.
Explore the human behaviour lexicon, the community impact section, and in particular the AI in community services impact area for more on what algorithmic accountability means for the sector.