Goodhart’s Law AI governance: why measuring AI governance compliance produces optimised metrics rather than genuine safe practice in nonprofits
Goodhart’s Law AI governance risk describes what happens when organisations use metrics to demonstrate responsible AI use: the metrics become targets, and once they become targets, they cease to be good measures of what they were designed to track. Established by economist Charles Goodhart in 1975 and extended by anthropologist Marilyn Strathern, Goodhart’s Law is directly relevant to nonprofit AI governance because community organisations under funder and regulatory pressure will naturally optimise their visible AI governance indicators rather than their actual AI governance practice. This behavioural risk connects to security theatre, normalisation of deviance, and legitimacy borrowing in the NFH Governance and Accountability theme, and underpins the NFH Behaviour Risk Card framework.
Goodhart’s Law
When a measure becomes a target, it ceases to be a good measure. AI governance metrics designed to demonstrate responsible use quickly become things to optimise rather than genuine indicators of safe practice.
Goodhart’s Law AI governance: why measuring AI governance compliance produces optimised metrics rather than genuine safe practice in nonprofits
Definition
What is Goodhart’s Law in AI Governance?
Goodhart’s Law states that when a measure becomes a target, it ceases to be a good measure. The relationship between measurement and the underlying reality it was designed to capture breaks down once people know they are being assessed against it. Behaviour shifts toward producing the measure, not toward producing the thing the measure was intended to reflect.
This is not a failure of intent. Organisations and individuals respond rationally to the incentive structures they face. When performance is assessed through specific indicators, optimising those indicators is the rational response. The problem lies in the assumption that improving the indicator improves the underlying reality, an assumption that Goodhart’s Law shows to be systematically unreliable.
Strathern’s Extension
Marilyn Strathern sharpened the principle into its most quoted form: when a measure becomes a target, it ceases to be a good measure. Her analysis of audit culture in British universities showed how formal evaluation processes produce organisations skilled at satisfying evaluation criteria rather than organisations genuinely improving in the dimensions those criteria were designed to assess. The audit produces auditability, not quality.
Research Origin
Goodhart (1975) identified the principle in the context of monetary policy: statistical regularities used as targets for policy control tend to collapse once pressure is applied to them. The measure works as an indicator only when it is observed passively. Active intervention to change the indicator disrupts the underlying relationship the indicator was tracking.
Strathern (1997) extended the principle into organisational and social contexts, demonstrating through her analysis of higher education audit culture that the proliferation of performance measures does not produce better performance. It produces better performance on measures. The distinction is consequential. Organisations become skilled at satisfying external scrutiny in ways that are increasingly disconnected from the internal realities that scrutiny was intended to assess.
Muller (2018) applied the principle comprehensively across sectors including healthcare, education, policing, and business, documenting the consistent pattern: metric fixation produces gaming, distortion of priorities, short-termism, and the crowding out of the professional judgement that effective practice actually requires. The organisations most subject to measurement become, over time, the least able to exercise the discretion that good outcomes depend on.
In AI Contexts
How Goodhart’s Law Shows Up in AI Governance
AI governance frameworks typically rely on measurable indicators: the existence of a policy, the number of staff trained, the frequency of governance reviews, the percentage of AI outputs that received sign-off before use. Each of these is a reasonable proxy for genuine governance when first adopted. Each becomes a target once funder requirements, board reporting, or regulatory expectations attach to it.
Optimising the Visible
Once these become targets, organisations optimise them. Training completion rates rise because completion is tracked, not because training changes behaviour. Sign-off rates increase because sign-off is logged, not because review is substantive. Policy documents become more detailed because detail signals effort to external reviewers, not because detail improves practice. Goodhart’s Law AI governance risk is the process by which governance frameworks produce the appearance of governance while the underlying risk picture remains unchanged or worsens.
Why It Matters for Nonprofits
The Specific Risk for Community Organisations
Community organisations face particular Goodhart pressure because their governance is observed by multiple external audiences simultaneously: funders, peak bodies, regulators, and communities. Each audience may use different indicators to assess responsible AI use. Satisfying all of them requires significant governance activity, and the rational response to that activity load is to produce the indicators efficiently rather than to address the underlying risks thoroughly.
When Metrics Displace Judgement
Muller’s analysis is directly relevant here: metric fixation crowds out professional judgement. For community organisations whose most important governance asset is the contextual knowledge and ethical judgement of experienced staff, a governance framework that substitutes measurable compliance for professional discretion actively degrades the organisation’s capacity to govern well. The metrics improve. The governance weakens. The gap between the two is invisible in any report that relies on the metrics to assess governance quality.
Goodhart’s Law AI governance risk describes what happens when organisations use metrics to demonstrate responsible AI use: the metrics become targets, and once they become targets, they cease to be good measures of what they were designed to track. Established by economist Charles Goodhart in 1975 and extended by anthropologist Marilyn Strathern, Goodhart’s Law is directly relevant to nonprofit AI governance because community organisations under funder and regulatory pressure will naturally optimise their visible AI governance indicators rather than their actual AI governance practice. This behavioural risk connects to security theatre, normalisation of deviance, and legitimacy borrowing in the NFH Governance and Accountability theme, and underpins the NFH Behaviour Risk Card framework.
Observable Signals
What to Watch For
These patterns suggest Goodhart’s Law may be shaping AI governance in your organisation. They are signals worth naming and discussing, not a definitive checklist.
- AI governance metrics consistently meet or exceed targets while staff and managers privately describe actual AI practice as largely ungoverned, suggesting the metrics are being optimised rather than the underlying practice.
- Governance activity increases around reporting periods and decreases between them, indicating that external scrutiny rather than internal risk management is driving governance behaviour.
- Staff complete AI governance training and sign off on AI governance policies without being able to describe how either has changed their specific day-to-day AI use decisions.
Goodhart’s Law AI governance: why measuring AI governance compliance produces optimised metrics rather than genuine safe practice in nonprofits
Goodhart’s Law AI governance risk describes what happens when organisations use metrics to demonstrate responsible AI use: the metrics become targets, and once they become targets, they cease to be good measures of what they were designed to track. Established by economist Charles Goodhart in 1975 and extended by anthropologist Marilyn Strathern, Goodhart’s Law is directly relevant to nonprofit AI governance because community organisations under funder and regulatory pressure will naturally optimise their visible AI governance indicators rather than their actual AI governance practice. This behavioural risk connects to security theatre, normalisation of deviance, and legitimacy borrowing in the NFH Governance and Accountability theme, and underpins the NFH Behaviour Risk Card framework.
External References
- Goodhart, C.A.E. (1975). Problems of monetary management: The UK experience. In A.S. Courakis (Ed.), Inflation, Depression, and Economic Policy in the West. Springer.
- Strathern, M. (1997). Improving ratings: Audit in the British university system. European Review, 5(3), 305-321.
- Muller, J.Z. (2018). The Tyranny of Metrics. Princeton University Press.
Related Lexicon Entries
From theory to practice
Understanding the risk is the first step.
The NFH Toolkit gives your organisation practical tools to test whether AI governance activity is changing actual practice, rather than producing indicators that satisfy external scrutiny while leaving underlying risk unaddressed. The Behaviour Risk Cards name this pattern in language your whole team can use.
Not For Humans: Because AI’s greatest risk isn’t technical, it’s Human.