13.2 Data Products, Platforms, and Federated Governance: Autonomy Must Be Interoperable

Data products have been released independently across domains, yet consumers must relearn every set of identity keys, time frameworks, and permission workflows, autonomy has become a new island of isolation.

The first domain each released a "product sheet," but they differed vastly in format, identity keys, temporal semantics, and permission request methods. Consumers now have to piece together three separate sheets, making the process slower than before. The issue isn't domain autonomy; it's that autonomy lacks a common interface, and the platform hasn't made the right path the default.

This lesson's objectives

• Define the interfaces, SLOs, and lifecycle of data products;
• The self-service pathway on the design platform;
• Convert federal rules into templates, policies, and validation evidence;
• Measure migration by consumer outcomes, not by product volume.

1. Data products deliver consumable capabilities

A product specification must at least include:

id: logistics.confirmed-supply-events
owner: logistics-domain
purpose: confirmed supply movement for operational and analytical use
interfaces:
  batch: table://prod/logistics/confirmed_supply_events/v2
  stream: topic://prod/logistics/confirmed-supply-events/v2
grain: one row/event per confirmed supply movement
keys: [supply_event_id]
event_time: confirmed_at
available_time: ingested_at
schema_contract: registry://logistics/supply-event/v2
quality_slos:
  freshness: consumer-agreed target
  completeness: based on expected outposts
classification: internal-sensitive
support: on-call route and issue tracker
deprecation: notice window and migration policy

Specific SLOs cannot be copied from templates; they must be negotiated between consumers based on deadlines, failure costs, and capacity. Whether an external SLA includes breach consequences depends on the organizational relationship.

2. The Eight Core Capabilities of a Product

• Discoverable: directories, names, terms, and examples;
• Readable: granularity, time, units, NULL, and state semantics;
• Trustworthy: Quality results, event status, and authentication;
• Addressable: Stable asset ID and interface;
• Interoperability: shared identity, time, schema, and policy agreements;
• Security: Classification, purpose, least privilege, and auditing;
• Valuable: Clearly defines consumers, decision-makers, and feedback mechanisms;
• Maintainable: ownership, support, versioning, cost, and retirement.

You don’t need to promote every internal temporary table into a product. The more products you have, the higher the support, catalog, and compatibility costs become.

3. Contract Separation of Syntax and Semantics

Schema contract constraints include types, nullability, enumerations, and compatibility; semantic contract specifies granularity, units, time, and business rules; service contract defines freshness, availability, support, and changes.

Compatibility goes beyond just the schema:

• duration changed from minutes to seconds, type unchanged but semantics are broken;
• State done is now interpreted as "Settlement Completed," enumeration unchanged;
• Historical data reinsertion alters past metrics, but queries can still run.

Therefore, change detection requires machine rules and owner/consumer review.

Platforms Should Also Be Operated as Products

Platform consumers are the domain product teams. The platform team should observe how many steps it takes to go from code to a secure release, and where failures occur, rather than just delivering a collection of infrastructure components.

A paved road can provide:

Product Templates and Warehouses
Schema/Contract registration and compatibility checking
Deployment, scheduling, and environment isolation
Quality SLI, Logging, Lineage, and Directory Registration
Identity, access control, and auditing
Cost, Quotas, and Resource Recommendations
Workflows for events, rollbacks, and retirement

The default path should be safe and low-friction, while still preserving exception extension points. Forcing all scenarios to use the same computation engine would make the platform restrictive; allowing unlimited freedom would undermine self-service and interoperability.

5. How Federal Computing Governance Is Put Into Practice

Common rules can be layered:

Globally Non-Negotiable

Identity authentication, sensitive data classification, minimum audit fields, significant use restrictions, and incident response.

Interoperability Standards

Asset ID, timestamp format, schema version, generic entity key, and contract agreement.

Domain Rules

Business enumerations, quality thresholds, release frequency, internal models, and support details.

Implement the rules as:

• Templates and default configurations;
• Schema/policy validation in CI;
• Access and category gatekeeping at time of release;
• Runtime monitoring and auditing;
• Exceptions with owner, compensation control, and expiration date.

"Computational governance" doesn't automatically decide all policies. Ambiguous or value-conflicting cases still require human judgment, with the system providing evidence and carrying out decisions.

6. Cross-Domain Identity and Composition

The product can function correctly on its own, but fails to integrate properly due to inconsistencies among physical keys, event timestamps, and slowly changing dimensions. Prioritize building a small set of shared capabilities:

• Global/mappable entity identifier;
• Master data and identity resolution;
• Event time, available time, and time zone conventions;
• Reference data version;
• Composition and attribution of cross-domain metrics.

Shared standards should address real consumer needs, not be created solely for the sake of unification, resulting in enterprise-wide large models that go unused.

7. Costs and Incentives

The domain bears production costs but may reap benefits from other teams. If owners are only evaluated on costs, they'll reduce sharing; if the platform is completely free and invisible, waste will result.

Needed:

• Display production, storage, and consumption costs;
• Provide a shared budget for cross-domain public goods;
• Incorporate reliability and consumer outcomes into domain goals;
• Retire low-value, consumer-unnecessary products promptly;
• Don't use query volume as the sole metric of value.

8. Measuring Whether True Improvement Has Occurred

A more meaningful metric than "how many data products":

• The time when consumers first realize they're using it successfully;
• Contract change lead time and destructive change rate;
• SLOs, event recovery, and recurring events;
• Owner/Consumer Issue Response Time;
• The number of manual mappings required for cross-domain integration;
• Platform paved road usage rate and escape reasons;
• Active products, non-consumer products, and retirement dates;
• The total cost of each business outcome.

Metrics must be compared to the pre-migration baseline and safeguards must be put in place to prevent domains from optimizing numbers by splitting products or hiding events.

9. Progressive Migration

1. Pick a consumer-facing cross-domain value stream
2. Clearly define the domain owner, product interfaces, and the current baseline
3. Fill in the three most painful steps in the platform release process
4. Establish a minimum global policy and exception mechanism
5. Run both old and new interfaces to verify semantics and SLO
6. Consumers migrate and provide feedback
7. Retire old paths, review organizational and platform gaps
8. Extend into the next domain only after the pattern is reusable

Don't preset generic phases of 6, 12, or 24 months. The pace depends on the existing platform, team capabilities, risk factors, and scope of migration.

Common Misconceptions

• A table with an owner is a data product: it's still missing interfaces, semantics, SLOs, support, and a lifecycle.
• Self-service means no platform support: The platform handles turning complex controls into usable capabilities.
• Federal means all rules are optional: Global baselines and interoperability agreements still need to be enforced.
• The more products, the more successful the transition: Over-splitting increases consumption and maintenance costs.

Practice

1. To enrich event product semantics, services, and security contracts.
2. Draw the paved road from the domain team submitting code to product release and identify manual bottlenecks.
3. Divide the three governance policies into global, interoperability, and domain layers, then define execution points.
4. Select five consumer outcome metrics for the first pilot, rather than product quantities.

Summary

Data Mesh autonomy must be achieved through stable product APIs, shared platform capabilities, and enforceable governance to enable interoperability. True success isn't about changing the organizational chart; it's about domains delivering faster and more securely, and consumers relying less on oral knowledge and manual coordination.

The final chapter dives into privacy engineering: regardless of whether data is centrally or locally managed, control over the risks to individuals caused by collection, linkage, access, retention, and dissemination must be ensured.