Epistemic justice AI governance: how AI systems inherit and amplify the discounting of marginalised community knowledge in not-for-profit, NGO, and community organisations in Australia
Epistemic justice AI governance describes the risk that AI systems adopted by not-for-profit organisations, NGOs, and community organisations systematically discount or exclude the knowledge, experience, and testimony of marginalised communities, reproducing existing patterns of epistemic injustice through computational means. Drawing on Miranda Fricker’s 2007 foundational framework establishing testimonial and hermeneutical injustice as the two primary mechanisms through which knowledge is unjustly discounted, Abeba Birhane’s 2021 relational ethics analysis of how algorithmic systems embed and amplify epistemic injustice, and Kristie Dotson’s 2012 analysis of epistemic oppression and the silencing of marginalised knowers, epistemic justice AI governance risk is directly relevant to NFPs and community organisations whose practice depends on centring the knowledge and experience of communities that mainstream data systems have historically marginalised. When AI tools are trained on data that does not represent those communities, or used in ways that substitute computational outputs for relational and experiential knowledge, the organisation risks reproducing the very epistemic inequities its mission exists to address. Epistemic justice AI governance connects to encoded bias, social license to operate, mechanical objectivity, and psychological safety in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for not-for-profit and community organisations across Australia.
Epistemic Justice
Whose knowledge counts? AI systems do not answer that question neutrally. They inherit the answer already embedded in their training data, which reflects whose knowledge has historically been recorded, valued, and preserved. For community organisations whose practice depends on centring marginalised knowledge, that inheritance is a governance problem.
Epistemic justice AI governance: how AI systems inherit and amplify the discounting of marginalised community knowledge in not-for-profit, NGO, and community organisations in Australia
Definition
What is Epistemic Justice in AI Governance?
Epistemic justice concerns the fair distribution of epistemic goods: being believed, being heard, having one’s knowledge treated as credible and valuable. Epistemic injustice occurs when someone is wronged specifically in their capacity as a knower. Fricker identified two primary mechanisms. Testimonial injustice occurs when a speaker’s credibility is deflated because of their identity, causing their testimony to be dismissed or discounted. Hermeneutical injustice occurs when a gap in collective interpretive resources means a person cannot make sense of their own social experience, or cannot make that experience intelligible to others.
Both mechanisms operate in AI systems. A system trained predominantly on data from credentialled, formally recorded sources will systematically deflate the epistemic weight of knowledge that arrives through different channels: community testimony, lived experience, oral tradition, relational knowledge, and the kinds of contextual understanding that peer workers and frontline staff carry. That deflation is not a malfunction. It is the predictable output of a system built on unrepresentative foundations.
The Governance Consequence
When AI outputs are used to inform decisions about communities, the epistemic injustice embedded in those outputs becomes a governance injustice. Decisions shaped by AI systems that discount particular forms of knowledge will systematically produce worse outcomes for the communities whose knowledge was discounted. Those communities have no mechanism to challenge the process because the discounting is invisible inside a system that presents itself as objective.
Research Origin
Fricker (2007) developed the concept of epistemic injustice through careful philosophical analysis of how power shapes who gets to count as a knower. Her framework identified credibility deflation as a pervasive and often invisible harm: when a speaker’s identity causes their testimony to be taken less seriously than the content warrants, the harm is done before any specific claim is evaluated. Applied to AI governance, Fricker’s analysis predicts that systems trained on data from dominant cultural and institutional sources will reproduce those credibility hierarchies computationally, treating knowledge that arrives through non-dominant channels as lower-quality input regardless of its actual value.
Birhane (2021) brought epistemic justice theory directly into the analysis of algorithmic systems, arguing that AI reproduces relational injustices because it is built through relationships that are themselves shaped by power imbalances. Her relational ethics framework established that the harm of algorithmic injustice is not merely about incorrect outputs but about the erasure of ways of knowing that are not legible to systems built from dominant data sources. For community organisations, Birhane’s analysis means that adopting AI tools without examining whose knowledge those tools can and cannot represent is not a neutral technical decision. It is a decision about whose knowledge the organisation will centre in its practice.
Dotson (2012) examined epistemic oppression as a structural phenomenon, distinguishing it from individual acts of epistemic injustice. Her analysis showed that epistemic oppression operates through the systematic exclusion of certain knowers from the conceptual and methodological frameworks used to evaluate knowledge claims. This structural dimension is critical for AI governance: it means that the problem cannot be solved by including more diverse voices in individual decisions if the underlying system continues to use frameworks that render those voices less legible. Community organisations that add consultation processes without examining the epistemic foundations of the AI tools they use may be addressing the symptom while the structural exclusion continues.
In AI Contexts
How Epistemic Injustice Shows Up With AI
AI systems used in community service contexts inherit the epistemic hierarchies of their training data. A large language model trained predominantly on English-language, formally published, institutionally generated text will produce outputs that reflect that epistemic foundation. Knowledge that arrives through different channels — community narrative, oral tradition, peer testimony, relational expertise — is either absent from the training data or represented in ways that reduce rather than preserve its character.
When AI Replaces Relational Knowledge
The governance risk intensifies when AI outputs begin substituting for the relational knowledge that community workers hold. A case summary generated by AI draws on whatever data has been formally recorded. It cannot draw on the contextual understanding a worker has built through sustained relationship with a client and community. When the AI summary informs a decision that the worker’s fuller knowledge would have shaped differently, an epistemic injustice has occurred: the less complete knowledge has displaced the more complete one because it arrived in a more legible form.
This pattern is particularly consequential in assessment contexts, where AI-generated risk scores, eligibility determinations, or need classifications carry decision authority that community workers find difficult to challenge. Challenging an AI output requires a worker to argue that their relational knowledge outweighs a computational one, in an organisational environment that may not have frameworks for making that argument.
In the Field: Anthropic’s 2026 analysis of labour market impacts found that workers in roles most exposed to AI displacement skew female, are typically educated, and occupy knowledge-intensive positions in community-facing sectors. The knowledge these workers carry — relational, contextual, culturally informed — is precisely the knowledge that AI systems are least equipped to represent. As those roles face structural disruption, the epistemic resources they hold are at risk of being lost from organisations before governance frameworks have been built to recognise and protect them. Anthropic Labour Market Impacts research ↗
Hermeneutical Gaps in AI Systems
Fricker’s concept of hermeneutical injustice — the gap between lived experience and the conceptual tools available to describe it — maps directly onto AI limitations. AI systems cannot produce outputs for experiences that are not represented in their training data. Communities whose experiences are structurally underrepresented produce a hermeneutical gap in any AI system applied to their contexts. The system does not flag that gap. It produces output anyway, and that output carries the authority of computational objectivity while containing the blind spots of unrepresentative foundations.
Why It Matters for Not-For-Profits
The Specific Risk for Community Organisations
Many not-for-profit and community organisations exist precisely because mainstream systems have failed to adequately serve particular communities. Their practice is grounded in the recognition that those communities hold knowledge about their own circumstances that mainstream systems cannot access or adequately represent. AI adoption that reproduces the epistemic hierarchies of mainstream systems risks undermining the foundational value proposition of the organisation.
First Nations Knowledge and Data Sovereignty
For organisations working with First Nations communities, epistemic justice in AI governance intersects directly with data sovereignty. First Nations knowledge systems operate through frameworks that mainstream data collection has historically excluded, appropriated, or misrepresented. AI systems trained on mainstream data sources do not merely lack First Nations knowledge. They may actively misrepresent it through the application of incompatible interpretive frameworks. Organisations that adopt AI tools without examining this dimension are not taking a neutral position on First Nations data sovereignty. They are making a governance choice about whose knowledge frameworks their practice will depend on.
Australian regulatory context: The National Indigenous Australians Agency (NIAA) Framework for Governance of Indigenous Data establishes principles for community engagement, consent, cultural protocols, and collective rights when First Nations data is used in AI systems. The NIAA has also published an AI Transparency Statement documenting how it governs AI use in relation to First Nations communities. Both documents set expectations that community organisations working with First Nations peoples should consider when assessing AI governance obligations. NIAA Framework for Governance of Indigenous Data ↗ NIAA AI Transparency Statement ↗
Peer and Lived Experience Workforce
Peer workers and lived experience staff carry knowledge that is structurally irreplaceable: it derives from experience that cannot be systematically captured in training data. When AI tools are positioned as equivalent or superior sources of information about community experience, the epistemic value of that workforce is implicitly discounted. Organisations that do not actively protect the epistemic authority of lived experience knowledge within their governance frameworks risk reproducing the very marginalisation their workforce model was designed to address.
Epistemic justice AI governance describes the risk that AI systems adopted by not-for-profit organisations, NGOs, and community organisations systematically discount or exclude the knowledge, experience, and testimony of marginalised communities, reproducing existing patterns of epistemic injustice through computational means. Drawing on Miranda Fricker’s 2007 foundational framework establishing testimonial and hermeneutical injustice as the two primary mechanisms through which knowledge is unjustly discounted, Abeba Birhane’s 2021 relational ethics analysis of how algorithmic systems embed and amplify epistemic injustice, and Kristie Dotson’s 2012 analysis of epistemic oppression and the silencing of marginalised knowers, epistemic justice AI governance risk is directly relevant to NFPs and community organisations whose practice depends on centring the knowledge and experience of communities that mainstream data systems have historically marginalised. When AI tools are trained on data that does not represent those communities, or used in ways that substitute computational outputs for relational and experiential knowledge, the organisation risks reproducing the very epistemic inequities its mission exists to address. Epistemic justice AI governance connects to encoded bias, social license to operate, mechanical objectivity, and psychological safety in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for not-for-profit and community organisations across Australia.
Observable Signals
What to Watch For
These patterns suggest epistemic injustice may be operating through AI adoption in your organisation. They are signals worth naming and discussing, not a definitive checklist.
- AI-generated assessments, summaries, or recommendations are used to inform decisions about clients or communities without a structured process for workers to bring contextual, relational, or culturally specific knowledge that the AI system cannot access, effectively positioning computational outputs as the primary epistemic authority in decisions that require human knowledge to be made well.
- First Nations community members, people from culturally and linguistically diverse backgrounds, or people with lived experience of the issues the organisation addresses have not been consulted about how AI tools represent or affect their communities, meaning the organisation has no mechanism to identify hermeneutical gaps before those gaps produce harm.
- Peer workers or lived experience staff find that their knowledge is increasingly supplemented or replaced by AI outputs in practice, without any governance discussion about the epistemic implications of that substitution for the communities the organisation serves and for the workforce model the organisation has built its practice around.
Epistemic justice AI governance: how AI systems inherit and amplify the discounting of marginalised community knowledge in not-for-profit, NGO, and community organisations in Australia
Epistemic justice AI governance describes the risk that AI systems adopted by not-for-profit organisations, NGOs, and community organisations systematically discount or exclude the knowledge, experience, and testimony of marginalised communities, reproducing existing patterns of epistemic injustice through computational means. Drawing on Miranda Fricker’s 2007 foundational framework establishing testimonial and hermeneutical injustice as the two primary mechanisms through which knowledge is unjustly discounted, Abeba Birhane’s 2021 relational ethics analysis of how algorithmic systems embed and amplify epistemic injustice, and Kristie Dotson’s 2012 analysis of epistemic oppression and the silencing of marginalised knowers, epistemic justice AI governance risk is directly relevant to NFPs and community organisations whose practice depends on centring the knowledge and experience of communities that mainstream data systems have historically marginalised. When AI tools are trained on data that does not represent those communities, or used in ways that substitute computational outputs for relational and experiential knowledge, the organisation risks reproducing the very epistemic inequities its mission exists to address. Epistemic justice AI governance connects to encoded bias, social license to operate, mechanical objectivity, and psychological safety in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for not-for-profit and community organisations across Australia.
External References
- Fricker, M. (2007). Epistemic injustice: Power and the ethics of knowing. Oxford University Press.
- Birhane, A. (2021). Algorithmic injustice: A relational ethics approach. Patterns, 2(2), 100205.
- Dotson, K. (2012). A cautionary tale: On limiting epistemic oppression. Frontiers: A Journal of Women Studies, 33(1), 24-47.
Related Lexicon Entries
From theory to practice
Understanding the risk is the first step.
The NFH Toolkit gives your organisation practical tools to assess the social license implications of AI adoption, build community consultation processes that are proportionate to your context and capacity, and establish transparency practices that protect the trust relationships your organisation depends on. The Behaviour Risk Cards name this pattern in language your whole team can use.
Concerned about how AI is affecting your community or the organisations that serve them?
Not For Humans: Because AI’s greatest risk isn’t technical, it’s Human.