14.4 Access, Retention, and Compliance Operations: Permissions Will Eventually Expire, and Copies Must Be Found

The data platform has a data map and pseudonymized analysis tables, yet an audit finds that a former analyst still has access, emergency CSV exports never expire, and deletion requests remove rows from the primary database but leave copies in backups and feature tables.

Privacy controls need to become part of identity management, data lifecycles, supplier oversight, and incident operations so that they work beyond the assessment report.

Learning Objectives

• Combine RBAC, ABAC, and data policies to enforce least privilege;
• Manage authorization requests, reviews, expiration, and break-glass;
• Make retention and deletion cover derivatives, caches, backups, and downstream copies;
• Establish request-response, third-party management, and privacy incident processes.

1. Roles Are Just the Starting Point for Authorization

RBAC grants capabilities by role, making it easy to manage, but it easily leads to coarse-grained roles like "analysts can view all analytical data." ABAC can include:

• Data classification and domain;
• User team, training, location, and employment status;
• Purpose, ticket, environment, and device status;
• Row/column ranges, time windows, and output limits.

This Python standard-library example evaluates one authorization decision. Classification uses an explicit allowed set, rather than ordering strings such as public and sensitive. All attributes are assumed to come from trusted identity, grant, and asset-catalog services:

from datetime import datetime, timezone

def can_read(user, grant, asset, purpose, now):
    try:
        if now.utcoffset() is None or grant["expires_at"].utcoffset() is None:
            return False
        return (
            user["role"] == "analyst"
            and user["domain"] == "operations"
            and user["employment"] == "active"
            and grant["subject_id"] == user["id"]
            and grant["asset_id"] == asset["id"]
            and grant["action"] == "read"
            and grant["purpose"] == purpose == "approved_safety_analysis"
            and grant["ticket_status"] == "active"
            and grant["revoked"] is False
            and asset["classification"] in user["allowed_classifications"]
            and now < grant["expires_at"]
        )
    except (KeyError, TypeError, AttributeError):
        return False

user = {"id": "synthetic-analyst", "role": "analyst", "domain": "operations",
        "employment": "active", "allowed_classifications": {"pseudonymous"}}
asset = {"id": "mission_summary", "classification": "pseudonymous"}
purpose = "approved_safety_analysis"
grant = {"subject_id": user["id"], "asset_id": asset["id"], "action": "read",
         "purpose": purpose, "ticket_status": "active", "revoked": False,
         "expires_at": datetime(2026, 9, 5, 12, tzinfo=timezone.utc)}
now = datetime(2026, 9, 5, 11, tzinfo=timezone.utc)
print(can_read(user, grant, asset, purpose, now))
print(can_read(user, grant, asset, purpose, grant["expires_at"]))
print(can_read(user, {**grant, "revoked": True}, asset, purpose, now))
print(can_read(user, grant, {**asset, "id": "raw_locations"}, purpose, now))
# True, False, False, False (one per line)

The expiry boundary is exclusive, so a request exactly at expiry is denied. Missing required attributes or time zones also deny access. This demonstrates policy evaluation only; the server must authenticate, validate attribute types, verify approvals, enforce row/column scope, and apply the decision. A claimed purpose cannot substitute for a valid grant. Revocations must propagate promptly rather than waiting for long-lived caches to expire. Client-side filter_columns() is not access control; enforce policy at an unavoidable service or storage boundary and define how deny rules take precedence.

2. Permission Lifecycle

Application: Assets, Purpose, Scope, Duration, and Responsible Party
Approval: owner/Data Protection/Security involved by risk level
Distribute: minimum scope, short-term credentials, bound to identity
Usage: Query, export, and policy-decision tracking
Review: the owner periodically confirms the need for access
Revocation: resignation, position change, expiration, event, or purpose completion

Permanently shared accounts and static keys compromise accountability. Prioritize personal/workload identities, short-lived credentials, and automatic revocation.

3. Break-glass Is Scoped Emergency Access

Emergency access should:

• Be enabled only for predefined incident types;
• Require strong authentication and a recorded reason;
• Limit assets, operations, and duration;
• Send real-time notifications to on-duty personnel or owners;
• Preserve tamper-resistant audit records;
• Undergo independent post-incident review.

“role_admin = all” lacks purpose and has no time boundaries. Platform maintainers should not be defaulted to reading plaintext business data; control-plane permissions and data-plane permissions should be separated.

4. Encryption and Key Boundaries

Encryption in transit and at rest are foundational, but specific protocols, algorithms, modes, and key lengths should adhere to the organization's current standards and risk requirements, never hardcoding a universally applicable "TLS/AES-256" list into the curriculum.

Key controls include:

• Keys are generated, rotated, and audited by a dedicated system;
• Data keys and key-encryption keys are layered;
• Key administrator and data access responsibilities are separated;
• Backups, logs, temporary files, and exports are all protected;
• Decryption occurs at the smallest trusted boundary;
• The conditions for key revocation/crypto-shredding have been verified.

Encryption cannot prevent authorized applications from making excessive queries, so purpose, output, and behavior controls are still necessary.

5. Derive Retention from Purpose and Obligations

Define for each record type:

Retention purpose and triggering event
Active period and archival period
statutory or contractual hold
Expiration actions: delete, anonymize, or review
Primary records, derivatives, caches, search indexes, and exports
Backup expiry and re-deletion after restore, before reads resume
Owner, execution evidence, and exception expiry

“Hold for seven years” is not a universal rule. Requirements vary by jurisdiction, record type, and obligation; conflicts require legal and business owner documentation.

For example, GDPR Article 17(3) includes erasure exceptions for legal obligations and legal claims. Record the applicable basis, restricted scope, and review conditions; a hold does not authorize continued analysis. See the official GDPR text.

6. Deletion is a distributed workflow

Deletion requests or expiration policies may involve:

• Identity mappings and primary records;
• Event tables, wide tables, and data lake snapshots;
• Search indexing, caching, and queues;
• Features, training set, and model artifacts;
• Reports, exports, and third parties;
• Backup and disaster recovery copies.

Lineage identifies candidate downstream assets; data maps add manual and unintegrated paths. Workflows need idempotency, retries, scope evidence, and failure queues.

Resolve the minimum subject IDs and target scope needed for execution before removing identity mappings; deleting the mapping first can make downstream features impossible to locate. The deletion ledger must also prevent late events, replay, and backup restoration from reviving removed records, while limiting its own retention and access.

Backups often cannot be edited immediately per record. Limit retention, isolate recovery, and record tombstones; replay the deletion ledger after restoration and before reads resume. Cryptographic erasure requires accounting for relevant key and plaintext copies, without destroying shared keys still needed for records that must remain. Satisfaction of a specific obligation depends on the applicable framework and validated implementation.

A successful primary-database task does not necessarily close the request:

Target	State	Next action and evidence
Primary database	deleted	Record version and deletion verification
Feature cache	retry_pending	Retry with the same request ID; verify the old value cannot be read
Isolated backup	deferred_until_expiry	Record expiry, deletion-before-read controls, and the approved basis

Primary-database deletion is 100% complete here, but the request still has unfinished work. Each applicable target needs a verified result or a tracked exception with basis, deadline, and owner. Tell the requester the actual scope; entering a queue does not mean all copies have been deleted.

7. Model Deletion Requires Evidence

Individual data points can influence model parameters, but removing a single row from the training set doesn't mean the existing model has forgotten it. Optional strategies:

• Regularly retrain from updated training snapshots;
• Sharded training or reconstructible artifacts;
• Evaluate machine unlearning in suitable scenarios;
• Record the version relationship between the model and training data;
• When unable to verify forgetting, accurately state the scope.

Keep essential audit evidence and minimal deletion logs, avoiding the preservation of complete deleted data solely to prove deletion.

SISA research reduces some unlearning costs by organizing training to limit each datum’s influence. It requires that training structure and its artifacts; adding a deletion endpoint to an arbitrary existing model does not provide the same guarantee.

8. Propagating Individual Requests and Corrections

Under applicable laws, contracts, or organizational policies, requests for access, correction, deletion, restriction, objection, or portability may be required. Engineering processes should:

1. Verify the identity of the requestor to prevent disclosure to impersonators;
2. Confirm jurisdiction, scope, exceptions, and deadline;
3. Search the main record, aliases, and downstream entries;
4. Summarize into understandable output or implement corrections/limitations;
5. Notify the appropriate recipient;
6. Record the decision, execution evidence, and appeal path.

Don't collect excessive new identification during authentication, and don't assume all request types and time limits are the same globally.

9. Third-Party and Cross-Border Handling

Supplier evaluations can't end with a questionnaire. Record:

• Data, purpose, roles, and processing locations;
• Subprocessors and change notifications;
• Evidence of security and privacy controls;
• Event notifications, assistance requests, and audit rights;
• Retention, return, deletion, and evidence;
• Exit migration and access revocation;
• Applicable mechanisms required for cross-border/cross-domain scenarios.

Contracts cannot replace technical controls. Provide only necessary fields, with separate identities, short-lived grants, export restrictions, and ongoing logs.

10. Privacy Incident Response

Privacy incidents include unauthorized disclosure, as well as erroneous associations, over-collection, erroneous automated decisions, inability to delete, and purpose drift.

Detect and protect evidence
Limit access, sharing, or decision-making
Identify data, personnel, recipients, and time frames
Evaluate potential harm and applicable notification duties
Correct data, decisions, and downstream copies
Communicate with affected parties/regulators/partners (as applicable)
Verify control before recovery
Review and update the data map, policies, and drills

Notification thresholds and timelines vary with law and facts and are determined by the authorized legal/privacy officer; the engineering team is responsible for providing timely and accurate scope and timelines.

For example, GDPR Articles 33 and 34 distinguish controllers, processors, and communication to individuals. One 72-hour timer does not fit every incident and recipient. Record awareness time, risk assessment, and notification recipients separately.

11. Operational Metrics

• Expired/orphaned permissions and average authorization duration;
• Sensitive exports, anomalous queries, and break-glass reviews;
• Completion rate of expired data deletion versus failure queue;
• Individual request times for each stage and rework;
• Assets with no owner, no retention rules, and unknown recipients;
• Supplier deletion confirmation and permission revocation;
• Event detection, containment, scope determination, and remediation timelines.

Metrics should not reward hasty request closure. Sample completed requests to verify the output and the outcome for affected people.

Common Misconceptions

• Column-level filtering functions are essentially a permissions system: Users can bypass client-side code to access underlying data.
• Admins must be able to see all plaintext: Control plane maintenance and data access can be separated.
• Primary-database deletion finishes the request: Derivatives, exports, third parties, and recovery paths still need attention.
• One compliance checklist works globally: Obligations depend on jurisdiction, role, purpose, and data.

Practice

1. Refactor the coarse-grained analyst role into a policy with purpose, expiry, and classification.
2. Design issuance, real-time alerts, and post-incident review for break-glass events.
3. Draw a state machine showing a delete request that clears the cache, features, model, backup, and vendor.
4. Simulate an event involving erroneous identity association, list containment, correction, and notification evidence.

Summary

The core of privacy operations is giving power and data a finite lifespan: access is traceable and automatically expires, copies can be found and retired according to purpose, errors can propagate and be corrected, and events can be promptly contained and remedied.

From data collection through governance and privacy, the next volume moves into AI and machine learning, exploring how models learn, validate, and enter real-world decision-making.