Attribute substitution AI governance: how the mind replaces hard evaluative questions with easier ones when assessing AI outputs
Attribute substitution AI governance describes the cognitive process by which people unconsciously replace a difficult question with an easier one when making judgements under uncertainty. First systematically described by Kahneman and Frederick in 2002 and developed further in Kahneman’s 2011 synthesis, attribute substitution AI governance risk is directly relevant to how staff in community organisations evaluate AI outputs: the hard question of whether an AI recommendation is accurate and appropriate for this specific person in this specific context is routinely substituted with the easier question of whether the output looks reasonable and is expressed with confidence. Because AI tools consistently produce fluent, well-structured, confident outputs, the substitute question almost always produces a positive answer, regardless of whether the original hard question would have. Attribute substitution AI governance connects to automation bias, processing fluency effect, and coherence bias in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for nonprofits and community organisations in Australia.
Attribute Substitution
When a question is too hard to answer directly, the mind answers an easier one instead and does not notice the substitution. Staff evaluating AI outputs face this constantly: the hard question of whether the output is right gets replaced by the easier question of whether it looks right.
Attribute substitution AI governance: how the mind replaces hard evaluative questions with easier ones when assessing AI outputs
Definition
What is Attribute Substitution AI Governance Risk?
Attribute substitution occurs when a person needs to assess a target attribute that is difficult to evaluate and instead, without awareness, substitutes a related attribute that is easier to assess. The judgment feels like it addresses the original question. In practice, it addresses a simpler proxy for it.
The substitution is not deliberate. It happens automatically, as part of the fast, intuitive processing that handles most everyday cognition. The person is not aware they have switched questions. They experience their answer as a response to the original question, not to the substitute. This is what makes attribute substitution particularly difficult to counter: it operates below the threshold of conscious deliberation.
The Target and the Heuristic Attribute
Kahneman and Frederick distinguished between the target attribute, the quality you actually want to assess, and the heuristic attribute, the quality you end up assessing instead. In AI governance contexts, the target attribute is accuracy, appropriateness, and fitness for purpose. The heuristic attribute is typically fluency, confidence, and surface plausibility. AI tools are optimised to produce outputs that score well on the heuristic attributes. They are not optimised for the target attributes that matter for governance.
Research Origin
Kahneman and Frederick (2002) provided the first systematic account of attribute substitution as a general mechanism underlying heuristic judgment. They argued that most of the classic heuristics identified in earlier research, including representativeness, availability, and anchoring, are instances of a common underlying process: a difficult target attribute is assessed by substituting an accessible heuristic attribute. The substitution produces systematic and predictable errors whenever the heuristic attribute diverges from the target attribute.
Kahneman (2003) extended the framework in his account of the two systems of cognitive processing: fast, automatic, associative System 1 processing, which is responsible for attribute substitution, and slow, deliberate, rule-governed System 2 processing, which can override System 1 but requires effort and attention. Critically, System 2 does not routinely monitor System 1 outputs for accuracy. It endorses them unless there is a specific signal to do otherwise.
Kahneman (2011) synthesised decades of research on intuitive judgment, placing attribute substitution within a comprehensive account of when intuition is reliable and when it systematically misleads. His analysis identified the conditions under which System 2 override is most likely to fail: time pressure, cognitive load, high confidence in the intuitive answer, and the absence of feedback about past errors. These are precisely the conditions of AI-assisted decision-making in community organisations under pressure.
In AI Contexts
How Attribute Substitution Shows Up With AI
Evaluating an AI output properly requires asking a hard question: is this accurate, does it apply to this specific situation, are there important factors it has missed, and is it appropriate for this person in this context? These questions require effort, contextual knowledge, and time.
The Fluency Substitution
Instead, staff typically assess a much easier question: does this output look professional, is it well-structured, and does it sound confident? AI tools are exceptionally good at producing outputs that pass this easier test. The language is fluent, the structure is clear, and the tone is assured. Attribute substitution means that passing the easy test is experienced as passing the hard one, even though the two tests measure very different things.
Furthermore, AI outputs arrive without the visible uncertainty that characterises human expert judgment. A practitioner reasoning through a complex case will hedge, qualify, and express doubt. An AI output presents a clean, confident answer. That confidence is a heuristic attribute, not evidence of accuracy. Attribute substitution converts it into evidence of accuracy anyway.
Why It Matters for Nonprofits
The Specific Risk for Community Organisations
Community organisations use AI to support decisions with direct consequences for vulnerable people: assessment recommendations, communication with clients, case planning, and resource allocation. In these contexts, the gap between the easy question and the hard question is not trivial. An AI output can be fluent, well-structured, and completely wrong about the specific person or situation it is being applied to.
When the Override Fails
Kahneman’s research identified the conditions under which deliberate System 2 override of System 1 substitution is least likely: time pressure, high cognitive load, and confidence in the intuitive answer. These conditions describe the standard operating environment of staff in community organisations using AI tools under service delivery pressure. The override that would catch the substitution is precisely the override that is most likely to fail.
Governance frameworks that depend on staff critically evaluating AI outputs are, in effect, depending on System 2 override. Making that override more likely requires more than good intentions: it requires structural conditions, including adequate time, reduced cognitive load, and explicit prompts to interrogate AI outputs, that most community organisations do not currently build into their AI workflows.
Attribute substitution AI governance describes the cognitive process by which people unconsciously replace a difficult question with an easier one when making judgements under uncertainty. First systematically described by Kahneman and Frederick in 2002 and developed further in Kahneman’s 2011 synthesis, attribute substitution AI governance risk is directly relevant to how staff in community organisations evaluate AI outputs: the hard question of whether an AI recommendation is accurate and appropriate for this specific person in this specific context is routinely substituted with the easier question of whether the output looks reasonable and is expressed with confidence. Because AI tools consistently produce fluent, well-structured, confident outputs, the substitute question almost always produces a positive answer, regardless of whether the original hard question would have. Attribute substitution AI governance connects to automation bias, processing fluency effect, and coherence bias in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for nonprofits and community organisations in Australia.
Observable Signals
What to Watch For
These patterns suggest attribute substitution may be shaping how staff evaluate AI outputs in your organisation. They are signals worth naming and discussing, not a definitive checklist.
- Staff describe accepting AI outputs using language that references surface quality rather than substantive accuracy: the output “looked good,” “sounded right,” or “seemed professional,” without reference to whether the content was correct or appropriate for the specific situation.
- Critical evaluation of AI outputs is most likely to be skipped under time pressure or high workload, suggesting that the evaluation is dependent on deliberate effort that competes directly with other demands rather than being embedded in the workflow itself.
- Staff find it difficult to articulate what a critical evaluation of an AI output would actually involve, indicating that the governance expectation to “review AI outputs” has not been translated into specific, actionable criteria for what review means.
Attribute substitution AI governance: how the mind replaces hard evaluative questions with easier ones when assessing AI outputs
Attribute substitution AI governance describes the cognitive process by which people unconsciously replace a difficult question with an easier one when making judgements under uncertainty. First systematically described by Kahneman and Frederick in 2002 and developed further in Kahneman’s 2011 synthesis, attribute substitution AI governance risk is directly relevant to how staff in community organisations evaluate AI outputs: the hard question of whether an AI recommendation is accurate and appropriate for this specific person in this specific context is routinely substituted with the easier question of whether the output looks reasonable and is expressed with confidence. Because AI tools consistently produce fluent, well-structured, confident outputs, the substitute question almost always produces a positive answer, regardless of whether the original hard question would have. Attribute substitution AI governance connects to automation bias, processing fluency effect, and coherence bias in the NFH Lexicon, and underpins the NFH Behaviour Risk Card framework for nonprofits and community organisations in Australia.
External References
- Kahneman, D. and Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In Gilovich, T., Griffin, D. and Kahneman, D. (Eds.), Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge University Press.
- Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697-720.
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Related Lexicon Entries
From theory to practice
Understanding the risk is the first step.
The NFH Toolkit gives your organisation practical tools to build the structural conditions that make critical evaluation of AI outputs more likely, including the Behaviour Risk Cards that name this pattern in language your whole team can use.
Not For Humans: Because AI’s greatest risk isn’t technical, it’s Human.