Trust and cognition AI governance documents eight cognitive biases that shape how community organisation staff respond to AI outputs. From processing fluency and automation bias to confirmation bias and anthropomorphism, these entries explain why human oversight of AI systems fails in predictable ways and what organisations can do about it.
Trust and cognition AI governance for community organisations and not-for-profits
Trust and Cognition
How AI outputs are received, trusted, and acted on — and why capable people under ordinary pressure find it hard to slow down and scrutinise.
How trust and cognition affect AI governance in nonprofits and community sector organisations
Trust and cognition AI governance documents eight cognitive biases that shape how community organisation staff respond to AI outputs. From processing fluency and automation bias to confirmation bias and anthropomorphism, these entries explain why human oversight of AI systems fails in predictable ways and what organisations can do about it.
About this theme
The brain was not built to scrutinise AI
AI governance does not fail because people are careless. It fails because the cognitive shortcuts that make us effective under pressure were not designed for a world where fluent, polished text can be wrong.
None of the research behind this theme was written with AI in mind. The earliest studies predate modern computing. Some of the foundational work on how humans process trust, coherence, and authority predates it by generations. These are not theories about technology. They are theories about people, about the enduring, documented patterns in how human beings receive information, assign credibility, and decide when something is good enough to act on.
That is what makes them so directly relevant now. The Human/AI interface is new territory. It is evolving faster than the research that might describe it, and the behavioural consequences are still largely unmapped. What we do know is that AI does not create these cognitive patterns. It finds them already present and exacerbates them, reliably and at scale, in ways that earlier environments simply did not. AI is optimised for performance, output, and the appearance of completeness. The collision between those design priorities and the way humans actually process information is what this theme documents.
Research note: Mere Exposure Effect
Robert Zajonc’s 1968 research demonstrated that repeated exposure to a stimulus reliably increases liking and perceived quality of it, independent of any rational evaluation. The effect operates below conscious awareness. Staff who use AI tools daily do not simply become more comfortable with the workflow. They become more trusting of the outputs, not because the outputs improve, but because familiarity itself generates a sense of reliability that cannot be separated from accuracy.
For not-for-profits and community organisations, the conditions amplify the stakes. Staff are time-poor. Trust is foundational to organisational culture. Grant reports, case notes, community communications, and policy submissions carry real weight and are difficult to retract. These are not environments where a second look is always possible, which makes understanding why the first look tends toward acceptance so important.
Research on framing effects documents how the presentation of information shapes decisions independently of content, and AI is a remarkably effective framing tool. The availability heuristic adds a further dimension: options that AI does not surface simply do not get considered, because what does not come to mind easily does not get weighed.
Research note: Inattentional Blindness
In a widely cited study, expert radiologists reviewing chest X-rays failed to notice a small gorilla superimposed on the image. Not through carelessness, but because expert attention is directed, not global. AI creates a structurally similar problem in reverse: it produces output that looks exactly like what you were expecting, formatted to appear authoritative, whether or not it is correct.
Further theoretical grounding
These concepts inform this theme without warranting standalone entries. They are part of the research foundation this theme draws on.
How something is presented shapes what people do with it, independently of content. AI pre-frames every output before a human engages with it.
People reason from what comes to mind easily. Options that AI does not surface do not get considered, because they are not mentally available.
Humans impose story structure on events to make them feel explainable and inevitable. AI generates narratives fluently, which makes its accounts feel more true than the evidence warrants.
You may also be interested in
Behavioural Insights Team: EAST Framework
How behavioural science is applied in organisational and policy contexts
The Decision Lab: Automation Bias and AI
How over-reliance on automated systems develops in practice
ACNC Governance Standards
The obligations that extend directly to how AI is adopted and governed in registered charities
Eight entries
The research behind the theme
Each entry takes an established concept and connects it directly to AI adoption in not-for-profits and community organisations — what the research shows, how it appears in practice, and what to watch for.
Processing Fluency Effect
When something is easy to read, it feels more true. AI output is always easy to read — which means it always feels more credible than it may actually be.
Read entry Parasuraman & Manzey, 2010Automation Bias
When a system produces a recommendation, people tend to follow it — even when their own judgement would suggest otherwise. AI recommendations carry the same pull, often amplified by the appearance of computational authority.
Read entry Taleb, 2007 — Narrative FallacyCoherence Bias
A document that flows logically feels sound. AI is exceptionally good at producing documents that flow logically — whether or not the underlying logic is grounded in reality.
Read entry Thaler & Sunstein, 2008Default Effect
People accept what is already in front of them. When AI output arrives looking complete and ready to use, it becomes the default — and defaults rarely get challenged.
Read entry Weizenbaum, 1966 — with Horton & Wohl, 1956ELIZA Effect
People attribute understanding, intent, and empathy to AI systems that have none. In community service contexts — where relationships are the work — this misattribution carries particular risk.
Read entry Gestalt PsychologyGestalt Closure
When something looks complete, the brain treats it as complete. AI output arrives structured, formatted, and finished-looking — which is precisely the moment when gaps inside it stop being visible.
Read entry Wason, 1960 — with Nickerson, 1998Confirmation Bias
People seek evidence that confirms what they already believe. When AI outputs match existing expectations, they pass without scrutiny — and the errors most likely to cause harm are exactly the ones confirmation bias leaves unchallenged.
Read entry Epley, Waytz & Cacioppo, 2007Anthropomorphism
People attribute human qualities to AI systems that have none. Conversational AI is designed to trigger this response — and the relational trust it produces makes professional oversight increasingly difficult to maintain.
Read entryTrust and cognition AI governance resources for the Australian community sector
Trust and cognition AI governance documents eight cognitive biases that shape how community organisation staff respond to AI outputs. From processing fluency and automation bias to confirmation bias and anthropomorphism, these entries explain why human oversight of AI systems fails in predictable ways and what organisations can do about it.
All themes
Continue exploring the lexicon
From theory to practice
The lexicon is the foundation. Our resources are what you build with it.
Understanding why these behavioural patterns occur is the first step. Not For Humans resources give your organisation the practical infrastructure to identify which patterns are active, name them with shared language, and build governance commitments that hold under pressure.
Not For Humans: Because AI’s greatest risk isn’t technical, it’s Human.