What is surveillance creep in AI governance for community organisations?
Surveillance creep AI governance describes the process by which data collected for one purpose is incrementally repurposed for monitoring or risk assessment without revisiting consent. Community organisations and not-for-profits need to understand surveillance creep to protect the trust relationships their practice depends on and to meet their obligations under the Australian Privacy Act.
How does surveillance creep affect not-for-profit service delivery and client trust?
Surveillance Creep
Surveillance creep AI governance begins with a question organisations rarely ask: what else is this data being used for? Data collected for one legitimate purpose has a tendency to migrate. In community service contexts, intake information becomes risk profiling. Case notes become performance monitoring. Engagement records become compliance evidence. Each step is incremental and individually justifiable. Together, they transform a service relationship into a surveillance relationship without anyone making that decision explicitly.
Surveillance creep AI governance and the Australian Privacy Act framework
Definition
What is Surveillance Creep in AI Governance?
Surveillance creep describes the incremental expansion of data collection and use beyond the purpose for which it was originally gathered and consented to. The process is characterised by small, individually justifiable steps that cumulatively transform the nature of a data relationship without that transformation being explicitly decided or disclosed. Each step appears reasonable in isolation. However, the endpoint of the process, routine monitoring and risk assessment of people who consented only to service delivery, would not have been accepted if proposed at the outset.
In community organisations, surveillance creep typically follows the introduction of AI and data management tools that create new capabilities for pattern recognition, predictive scoring, and automated flagging. Data that was previously collected for administrative or service planning purposes becomes newly useful for monitoring individual behaviour, predicting risk, or assessing compliance. The organisation does not necessarily decide to surveil its clients. Instead, the capability exists, a justification presents itself, and the data is used. That sequence, repeated across multiple data types and decision contexts, constitutes surveillance creep.
The Consent Problem
The core governance issue in surveillance creep is that consent given for one purpose does not extend to repurposed uses of the same data, even when those uses are technically permitted by broadly worded privacy policies. Clients who disclosed personal information to access a housing service did not consent to that information being used to generate a risk score that determines their eligibility for future services. Clients who engaged with a mental health programme did not consent to their engagement patterns being monitored as a performance indicator. Surveillance creep AI governance requires organisations to assess not just whether data use is technically permissible, but whether it is consistent with the reasonable expectations of the people who provided the data.
How this differs from Consent Drift: Consent Drift describes what vendors do with data through expanding terms and conditions over time, moving the boundaries of permissible use without users actively agreeing to the change. Surveillance Creep describes what organisations themselves do: repurposing data they collected from clients for monitoring or risk assessment purposes beyond the original service context. Both involve the erosion of consent boundaries. However, Consent Drift is driven by vendor behaviour; Surveillance Creep is driven by organisational decisions about how to use the data they hold.
Research Origin
Marx (2006) analysed the expansion of surveillance in contemporary societies through his concept of soft surveillance: the collection of personal information through systems that individuals participate in voluntarily, often because participation is a practical necessity, but where the full scope of data use is not transparent at the point of collection. His analysis identified function creep as a central mechanism of surveillance expansion: the tendency for data collected for one purpose to be repurposed for monitoring, control, or risk management as technological capabilities and institutional incentives evolve. Marx established that surveillance creep does not require malicious intent. It requires only the coexistence of available data, new analytical capabilities, and institutional pressure to manage risk and demonstrate accountability. All three conditions are present in community organisations adopting AI tools.
The Privacy Act APP framework establishes the Australian legal context for surveillance creep. Australian Privacy Principle 3 governs the collection of solicited personal information and requires that organisations collect only information that is reasonably necessary for their functions or activities. Furthermore, APP 6 restricts secondary use and disclosure of personal information: organisations may use or disclose personal information for a secondary purpose only if the individual would reasonably expect that use, if they have consented, or if a specific exception applies. Together, these principles establish that the incremental repurposing of client data for monitoring or risk assessment purposes constitutes a prima facie APP compliance issue, regardless of whether the data was lawfully collected in the first instance.
The Privacy Act Review Report (2022) identified automated decision-making and data repurposing as priority concerns for reform. The Report recommended that organisations be required to provide clearer notification when personal information is used in automated decision-making that significantly affects individuals, and that the reasonable expectation test for secondary use be strengthened. For community organisations, the direction of reform reinforces the governance principle that technical permissibility is not the appropriate standard for assessing surveillance creep. The relevant question is whether the person whose data is being used would reasonably expect that use, and whether the relationship of trust that underlies the service context is being honoured or exploited.
In AI Contexts
How Surveillance Creep Shows Up With AI
Surveillance creep AI governance risks are present wherever community organisations use integrated data systems, case management platforms, or AI tools that aggregate client information across service interactions. These systems create analytical capabilities that did not exist when the data was originally collected, and those capabilities create new opportunities for repurposing that data in ways that were not anticipated or disclosed at the point of consent.
Common Surveillance Creep Pathways
In practice, surveillance creep in community organisations tends to follow recognisable pathways. Intake and assessment data collected for service planning is repurposed to generate predictive risk scores that determine service priority or eligibility. Engagement and attendance records collected for reporting purposes are repurposed to monitor client behaviour or assess compliance with programme requirements. Case notes collected to support continuity of care are repurposed to identify patterns that trigger automated alerts or referrals. In each case, the original consent context was service delivery. The repurposed use is monitoring and assessment.
AI tools accelerate this process because they make pattern recognition and predictive scoring technically straightforward. A case management system that previously required significant manual effort to analyse longitudinally can, with an AI layer, automatically surface risk scores, flag anomalies, and generate predictive assessments from the same data it has always held. The data did not change. The capability to use it for surveillance did.
The Aggregation Problem
Surveillance creep is amplified by data aggregation. Individual data points collected across different service interactions may each seem innocuous and clearly within the original consent scope. However, when aggregated and analysed together, they can reveal patterns and enable inferences that go substantially beyond what any individual disclosure authorised. An AI system that combines attendance records, engagement frequency, case note sentiment, and geographic movement data can produce a behavioural profile that no client consented to creating, from data that each individual service context legitimately collected.
Australian regulatory context: APP 3 requires collection of only information reasonably necessary for the organisation’s functions. APP 6 restricts secondary use to purposes within the reasonable expectations of the individual or where consent has been obtained. The OAIC’s Notifiable Data Breaches scheme additionally creates obligations where repurposed data use results in serious harm. Organisations should assess their AI tools against these principles, specifically examining whether any analytical or predictive functions use client data beyond the original collection purpose. OAIC APP 3 collection guidance ↗ OAIC Notifiable Data Breaches ↗
Why It Matters for Not-For-Profits
The Specific Risk for Community Organisations
Community organisations hold some of the most sensitive personal information that exists: disclosures made in the context of crisis, vulnerability, and trust. The therapeutic and service relationships through which that information is shared depend on clients believing that what they disclose will be used to help them, not to monitor, assess, or manage them. Surveillance creep directly threatens that belief, and therefore directly threatens the organisational capacity to serve the people who most need help.
The Trust Erosion Effect
Clients who become aware, or who suspect, that their disclosures are being used for purposes beyond the immediate service context will disclose less. They will engage less fully. They will present the version of themselves that generates the most favourable assessment rather than the version that most accurately reflects their circumstances. In this way, surveillance creep does not just create a compliance risk. It actively degrades the quality of the information the organisation needs to provide effective services, because the people most in need of those services have the most to lose from surveillance and adjust their behaviour accordingly.
This effect is particularly pronounced for communities with historical experience of state surveillance: First Nations communities, people with experience of the criminal justice system, and people from communities with experience of authoritarian governance. For these communities, the threshold at which surveillance creep damages trust is lower, and the consequences of that damage are more severe. Organisations working with these communities carry a heightened responsibility to examine how their AI tools use client data.
Funder and Reporting Pressures
Surveillance creep in community organisations is frequently driven by external accountability pressures rather than internal decisions. Funders increasingly require detailed outcome data, engagement metrics, and evidence of programme effectiveness. Reporting obligations create incentives to collect and analyse client data in ways that serve accountability rather than service delivery. These pressures are legitimate in themselves, but they create conditions where client data migrates from service use to performance management without that shift being explicitly governed. Recognising funder and reporting requirements as a surveillance creep risk is the first step toward managing it.
A governance starting point: For each AI function that uses client data, organisations should be able to answer two questions: what purpose was this data collected for, and is the current use of this data within the reasonable expectations of the person who provided it? Where the answer to the second question is uncertain, that uncertainty is itself a governance signal requiring attention before the function continues.
Surveillance creep AI governance describes the process by which data collected for one purpose is incrementally repurposed for monitoring or risk assessment without revisiting consent. Community organisations and not-for-profits need to understand surveillance creep to protect the trust relationships their practice depends on and to meet their obligations under the Australian Privacy Act.
Observable Signals
What to Watch For
These patterns suggest surveillance creep may be operating through AI adoption in your organisation. They are signals worth naming and discussing, not a definitive checklist.
- AI or data management tools are generating risk scores, behavioural flags, or predictive assessments from client data that was originally collected for service delivery or administrative purposes, and the organisation has not examined whether that secondary use is within the reasonable expectations of the clients who provided the data or consistent with the purpose stated at collection.
- Client engagement, attendance, or interaction data collected for programme reporting is also being used to monitor individual client behaviour, assess compliance, or inform decisions about service eligibility or priority in ways that were not disclosed to clients as a purpose of data collection.
- Workers notice that clients from communities with historical experience of state surveillance — including First Nations clients and clients with experience of the criminal justice system — are becoming less forthcoming in disclosures, or are presenting information in ways that appear shaped by concern about how it will be used rather than by what would most help them access appropriate support.
- The organisation’s privacy policy and client consent documentation describe data use in terms of service delivery and reporting, but the AI tools the organisation now uses have analytical capabilities that go beyond those described purposes, and the consent documentation has not been reviewed or updated to reflect what those tools actually do with client data.
Surveillance creep AI governance and the Australian Privacy Act framework
Surveillance creep AI governance describes the process by which data collected for one purpose is incrementally repurposed for monitoring or risk assessment without revisiting consent. Community organisations and not-for-profits need to understand surveillance creep to protect the trust relationships their practice depends on and to meet their obligations under the Australian Privacy Act.
External References
- Marx, G.T. (2006). Soft surveillance: The growth of mandatory volunteerism in collecting personal information. In T. Monahan (Ed.), Surveillance and security: Technological politics and power in everyday life. Routledge.
- Office of the Australian Information Commissioner. Australian Privacy Principles guidelines — Chapter 3: APP 3 collection of solicited personal information.
- Office of the Australian Information Commissioner. Notifiable Data Breaches scheme.
- Attorney-General’s Department. (2022). Privacy Act Review Report. Australian Government.
Related Lexicon Entries
From theory to practice
Understanding the risk is the first step.
The Not For Humans Toolkit gives your organisation practical tools to examine how client data is being used across your AI systems, identify where data has migrated beyond its original consent scope, and build governance practices that protect the trust relationships your service delivery depends on. The Behaviour Risk Cards name surveillance creep in language your whole team can use, and the Leadership Self Reflection Tool prompts the data purpose questions that tend to go unasked when new AI capabilities become available.
Concerned about how AI is affecting your community or the organisations that serve them?
Not For Humans: Because AI’s greatest risk isn’t technical, it’s Human.