14.1 Privacy Engineering and Data Mapping: Start by Reducing Unnecessary Processing

An audit finds that an external partner is collecting precise employee location trails. The data platform's first instinct is to "encrypt the database," but no one can explain which devices collect the locations, what decisions they inform, how many reports receive copies, or how long the data is retained. Security controls may prevent unauthorized access, but they do not answer whether the organization already collects and infers more than necessary.

Privacy, data protection, and information security are related but distinct concepts. Specific legal obligations depend on jurisdiction, role, data type, and purpose. This chapter explains engineering practices that support compliance; determining legal obligations requires analysis of the applicable framework and facts.

Learning Objectives

• Distinguish between privacy risks, security risks, and compliance obligations;
• Establish a data flow map covering collection, inference, sharing, and deletion;
• Minimize data processing through purpose limitation, necessity, and proportionality;
• Complete a privacy impact assessment and make a go/no-go decision before deploying in high-risk use cases.

1. Privacy Risks Go Beyond Data Leaks

Even if a system is never attacked, it can still generate:

• Persistent tracking that erodes people's reasonable freedom of action;
• Inference of new sensitive attributes from data collected for outdated purposes;
• Opaque profiling that distorts opportunities, treatment, or personal liberties;
• Incorrect data that cannot be corrected but continues to circulate;
• Group-level statistics that stigmatize or target small, vulnerable populations;
• Data centralization that enables internal personnel to observe beyond their authorized scope.

Security focuses on confidentiality, integrity, and availability; privacy also emphasizes predictability for people, autonomy, choice, dignity, and fairness of data processing. Compliance regulations set minimum obligations for a given jurisdiction, while ethical evaluations may demand higher standards, or may outright reject data practices as unacceptable.

2. Data classification should be combined with context

You can't just split fields by column name into PII/non_sensitive. The same value carries different risks in different contexts: combining a publicly available job title with an exact location and night shift times might uniquely identify someone.

For each data element record:

Subject/Group and Origin
Direct, indirect identifiers and linkable keys
Content and properties that may be inferred
Purpose of Collection and Permitted Uses
Processing roles and responsible parties
Storage location, region, replication, and recipient
Authorized users, automated decisions, and manual decisions
Retention/deletion and rights-request rules
Risk Level and Assessment Basis

Column names and regex scanning can help identify clues, but they miss free text, images, traces, and business codes and can generate false positives. The scanning process itself requires controlled access, minimal sampling, and auditing.

3. Map Processing Flows Beyond Databases

A data map traces data from source to final use:

Device collection
→ Edge caching
→ Message queue
→ Raw object storage
→ Cleaning / identity mapping
→ Features and reports
→ Decision systems / external recipients
→ Archiving, backup, and deletion

Each edge should be annotated with its purpose, transmission security, recipient, region, retention policy, and data version. Also document inferred information: the absence of "night shift health risk" in collected data does not imply that the model did not infer it from movement patterns and heart rate.

Lineage provides candidate technical paths but cannot replace information about unintegrated tools, manual exports, vendors, and processing purposes.

4. Purpose Limitation and Use Change

Purpose must be specific enough to assess compatibility. For example, "improve service" is too broad; "detect disconnections during shifts and trigger emergency response" provides clear constraints on fields, frequency, and data retention.

When a new use emerges, it must be re-evaluated against the following criteria:

• The relationship between the new use and the original purpose;
• The reasonable expectations and power dynamics of the affected individuals;
• The sensitivity of the data and any new inferences that may be drawn;
• The potential consequences for individuals and groups;
• Whether existing authorizations or legal bases adequately cover the new use;
• Whether new notices, user choices, or approvals are required;
• Whether anonymization, aggregation, or less data can be used to mitigate risk.

Encryption or pseudonymization can reduce certain risks, but it does not automatically make an incompatible use reasonable or lawful.

5. Data Minimization in System Design

Challenges to address:

• Is data collection absolutely necessary, or can events be generated and processed directly on the device?
• Do we need exact values, or are ranges or regions sufficient?
• Is continuous data collection required, or can we rely on short time windows or event-triggered sampling?
• Do we need data that can be linked to the same individual, or should each session use a randomly generated identifier?
• Should raw data be stored, or should it be aggregated and processed promptly?
• Does every team need a copy of the data?
• Can the model achieve its decision-making goals with fewer features?

The baseline for minimization is the clearly defined purpose, not "might be useful later." It's essential to measure the utility, group-level impact, and operational risks after removing specific fields.

For an on-shift lost-contact alert, a device could first detect timeout locally and upload only the alert, coarse area, and temporary session ID. An authorized rescue workflow could request short-lived precise location, with temporary caches cleared after the shift. Validate missed alerts and recovery delay if offline behavior could undermine safety. Decide whether to retain continuous trails from purpose and evidence, not cheap storage.

6. Consent is just one possible governance mechanism

In applicable frameworks, consent may come with strict conditions. Even without legal considerations, meaningful choice requires: information that is understandable, a real ability to decline, a mechanism for revocation that is enforceable, and proper handling of situations involving strong power imbalances.

Some processes rely on contracts, public duties, statutory obligations, or other forms of authorization rather than on consent; specific determinations must be made by individuals familiar with both the relevant laws and the factual context. Engineering systems must record the legal or organizational basis for processing, scope of application, version, and changes, rather than storing a perpetually valid consent=true in a database.

For processing within GDPR’s scope, Article 6 sets out several lawful bases, Article 5 addresses purpose and minimization, Article 25 covers design and default protection, and Article 35 requires a DPIA under its high-risk conditions. A consent boolean cannot describe the processing basis, and these provisions cannot be generalized to every jurisdiction. See the official GDPR text.

7. Privacy Impact Assessment

Conduct a structured assessment before deployment for high-risk scenarios, emerging technologies, large-scale monitoring, sensitive inferences, or significant automated decisions. Whether legal DPIA requirements apply must be determined based on jurisdiction.

Purpose, scope, and parties involved
Necessity and alternative approaches
Data flows, inferences, and retention periods
Affected individuals and their reasonable expectations
Risks of misuse, disclosure, error, and abuse of power
Severity, likelihood, and reversibility
Technical and organizational controls, along with residual risks
Stakeholder engagement and points of disagreement
Approvals, termination conditions, and review timelines

An assessment is not a checkbox to be ticked off in the release approval process. If residual risks remain high, the purpose is ambiguous, or controls cannot be verified, the conclusion may be to narrow the scope or defer implementation entirely.

8. Privacy by Default

Default settings should favor minimal data processing:

• New fields should not propagate downstream by default;
• New recipients should not have access rights by default;
• Any new use case must be explicitly registered;
• Data retention periods expire and trigger deletion or review;
• High-risk exports should be blocked or require approval by default;
• Precise location, public visibility, and long-term links should be disabled by default;
• Failures should not silently expand data collection.

Common Misconceptions

• Having encryption solves privacy: An authorization system can still perform excessive data processing.
• Scanning column names equals a data inventory: Context, inference, and copy chains remain unknown.
• Obtaining consent allows unlimited reuse: Purpose, power dynamics, and revocation mechanisms must still be addressed.
• Privacy assessments are solely written by legal teams: Engineering realities and actionable controls must be provided by product and technical teams.

Exercise

1. Create a data flow map from device to deletion, including two manually defined export paths.
2. Rewrite "for security" as specific, constraint-driven purposes involving field-level controls and retention periods.
3. Develop data-minimization options for precise location, continuous data collection, and long-term identity linkage.
4. Define go/no-go criteria and residual risk for a new inference use case.

Summary

Privacy engineering begins with understanding data flow: how data is generated, how it's linked, who it affects, and when it exits the system. Reducing unnecessary collection and inference is typically more reliable than adding post-hoc controls to large datasets.

The next lesson covers "de-identification": what risks are mitigated by masking, hashing, and tokenization, and why these techniques still cannot be easily considered anonymization.