14.4 Access, Retention, and Compliance Operation: Permissions Will Eventually Expire, and Copies Must Be Found

The Prediction Hall completed the data map and pseudonymized the analysis tables, yet during audit, it was discovered that a former analyst still had long-term permissions, emergency-exported CSV files had no expiration date, deletion requests were only processed for the primary database, and copies remained in backups and feature tables.

Privacy controls only remain in assessment reports if they don't enter identity, data lifecycle, supplier, and event operations.

This lesson's objectives

• Combine RBAC, ABAC, and data policies to enforce least privilege;
• Manage authorization requests, reviews, expiration, and break-glass;
• Keep and remove override derived, cached, backed up, and downstream;
• Establish request-response, third-party management, and privacy incident processes.

1. Roles Are Just the Starting Point for Authorization

RBAC grants capabilities by role, making it easy to manage, but it easily leads to coarse-grained roles like "analysts can view all analytical data." ABAC can include:

• Data classification and domain;
• User team, training, location, and employment status;
• Purpose, work order, environment, and equipment status;
• Row/column ranges, time windows, and output limits.

Strategy Overview:

allow read pseudonymous mission data
when user.role = analyst
and user.domain = operations
and purpose = approved_safety_analysis
and ticket.status = active
and asset.classification <= user.clearance
and current_time < grant.expires_at

This still requires clarity on source identity, attribute trustworthiness, default denial, and conflict priority. The client filter_columns() is not access control; enforcement should occur at the data service or storage layer, not bypassable.

2. Permission Lifecycle

Application: Assets, Purpose, Scope, Duration, and Responsible Party
Approval: owner/Data Protection/Security involved by risk level
Distribute: minimum scope, short-term credentials, bound to identity
Usage: Query, export, and strategy decision tracking
Reverification: the owner regularly confirms genuine needs
Revocation: resignation, position change, expiration, event, or purpose completion

Permanently shared accounts and static keys compromise accountability. Prioritize personal/workload identities, short-lived credentials, and automatic revocation.

3. Breaking glass doesn't mean admin full token

Emergency access should:

• Enabled only when defining event types;
• Strong authentication and justification;
• Limited assets, operations, and time-bound;
• Send real-time notifications to on-duty personnel or owners;
• Immutable audit;
• Independent post-incident review.

“role_admin = all” lacks purpose and has no time boundaries. Platform maintainers should not be defaulted to reading plaintext business data; control-plane permissions and data-plane permissions should be separated.

4. Encryption and Key Boundaries

Transmission and static encryption are foundational, but specific protocols, algorithms, modes, and key lengths should adhere to the organization's current standards and risk requirements, never hardcoding a universally applicable "TLS/AES-256" list into the curriculum.

More importantly:

• Keys are generated, rotated, and audited by a dedicated system;
• Data keys and key encryption keys layered;
• Key administrator and data access responsibilities are separated;
• Backups, logs, temporary files, and exports are all protected;
• Decryption occurs at the smallest trusted boundary;
• The conditions for key revocation/crypto-shredding have been verified.

Encryption cannot prevent authorized applications from making excessive queries, so purpose, output, and behavior controls are still necessary.

5. Retain the derivation from purpose and duty

Define for each record type:

Keep purpose and initial event
Active period and archival period
statutory or contractual hold
Expiration actions: delete, anonymize, or review
Master, derived, cache, search index, and export scope
Backup expiration and restore-then-delete mechanism
owner, executing evidence, and exception period

“Hold for seven years” is not a universal rule. Requirements vary by jurisdiction, record type, and obligation; conflicts require legal and business owner documentation.

6. Deletion is a distributed workflow

Deletion requests or expiration policies may involve:

• identity mapping with the master record;
• Event tables, wide tables, and data lake snapshots;
• Search indexing, caching, and queues;
• Features, training set, and model artifacts;
• Reports, export, and third-party;
• Backup and disaster recovery copies.

Lineage provides candidate downstreams, data maps complete non-automatic paths. Workflows need idempotency, retry, range evidence, and failure queues.

Backups can't always be modified on a per-item basis. To address this, shorten backup retention, restrict recovery access, record tombstones, and re-delete after recovery. Whether these measures satisfy specific legal obligations must be confirmed under applicable regulations.

7. You Can't Haphazardly Promise Deletion in a Model

Individual data points can influence model parameters, but removing a single row from the training set doesn't mean the existing model has forgotten it. Optional strategies:

• Regularly retrain from updated training snapshots;
• Sharded training or reconfigurable workloads;
• In suitable scenario assessment of machine unlearning;
• Record the version relationship between the model and training data;
• When unable to verify forgetting, accurately state the scope.

Keep essential audit evidence and minimal deletion logs, avoiding the preservation of complete deleted data solely to prove deletion.

8. Individual Requests and Corrections in Communication

Under applicable laws, contracts, or organizational policies, requests for access, correction, deletion, restriction, objection, or portability may be required. Engineering processes should:

1. Verify the identity of the requestor to prevent disclosure to imitators;
2. Confirm jurisdiction, scope, exceptions, and deadline;
3. Search the main record, aliases, and downstream entries;
4. Summarize into understandable output or implement corrections/limitations;
5. Notify the appropriate recipient;
6. Document the decision, evidence of execution, and appeal pathways.

Don't collect excessive new identification during authentication, and don't assume all request types and time limits are the same globally.

9. Third-Party and Cross-Border Handling

Supplier evaluations can't end with a questionnaire. Record:

• Data, purpose, roles, and processing locations;
• Child handlers and change notifications;
• Evidence of security and privacy controls;
• Event notifications, assistance requests, and audit rights;
• retain, return, delete, and verify;
• Termination of migration and revocation of access;
• Applicable mechanisms required for cross-border/cross-domain scenarios.

A contract cannot overcome technical limitations. Only provide essential fields, independent identity, short-term authorization, export restrictions, and continuous logging.

10. Privacy Incident Response

Privacy incidents include unauthorized disclosure, as well as erroneous associations, over-collection, erroneous automated decisions, inability to delete, and purpose drift.

Detect and protect evidence
Limit access, sharing, or decision-making
Identify data, personnel, recipients, and time frames
Evaluating potential harm and the applicability of notice obligations
Correct data, make decisions, and update downstream copies
Communicate with affected parties/regulators/partners (as applicable)
Verify control before recovery
Review and update the data map, strategies, and drills

Notification thresholds and timelines vary with law and facts and are determined by the authorized legal/privacy officer; the engineering team is responsible for providing timely and accurate scope and timelines.

11. Running Metrics

• Expired/orphaned permissions and average authorization duration;
• Sensitive exports, exception queries, and break-glass approvals;
• Completion rate of expired data deletion versus failure queue;
• Individual request times for each stage and rework;
• Assets with no owner, no retention rules, and unknown recipients;
• Supplier deletion confirmation and permission revocation;
• Event detection, containment, scope determination, and remediation timelines.

Metrics can't incentivize hasty request closures; sampling validates output correctness and impacted results.

Common Misconceptions

• Column-level filtering functions are essentially a permissions system: Users can bypass client-side code to access underlying data.
• Admins must be able to see all plaintext: Control plane maintenance and data access can be separated.
• Master database deletion completes: Derivative, export, third-party, and recovery paths remain.
• Global Compliance Checklist: Obligations depend on jurisdiction, role, purpose, and data.

Practice

1. Refactor the coarse-grained analyst role into a strategy with purpose, time limits, and categorization.
2. Design issuance, real-time alerts, and post-incident review for break-glass events.
3. Draw a state machine showing a delete request that clears the cache, features, model, backup, and vendor.
4. Simulate an event involving erroneous identity association, list containment, correction, and notification evidence.

Summary

The core of privacy-by-design is giving power and data a finite lifespan: access is traceable and automatically expires, copies are discoverable and can be exited based on purpose, errors can propagate and be corrected, and events can be promptly contained and remedied.

From data collection through governance and privacy, the next chapter moves into AI and machine learning, exploring how models learn, validate, and enter real-world decision-making.