10.2 Fairness Metrics and Group Evaluation: First, Ask Which Error Hurts Who Most

The error rate in the Northern Group is significantly higher, and the Prophecy Hall needs to determine whether the difference stems from sampling, labeling, baseline rates, or actual systemic harm.

The overall accuracy is 91%, but the North region only reaches 82%. This warrants investigation, yet we can't simply apply the rule that "a difference over 5% is unfair": sample size, label quality, baseline rates, and the consequences of error are all unknown. Fairness isn't a one-size-fits-all formula detached from context.

This lesson's objectives

• Break down predictions, labels, and error rates by group;
• Distinguish demographic parity, equal opportunity, equalized odds, and calibration;
• Use confidence intervals and cross-group analysis to avoid misjudgments from small samples;
• Link metric differences to actionable mitigation measures.

1. What kind of errors does accuracy hide

For binary classification, first report the confusion matrix for each group:

• true positive rate: the proportion of actual positives correctly identified;
• false negative rate: the proportion of true positives that are incorrectly classified as negatives;
• false positive rate: the proportion of true negatives incorrectly identified as positives;
• precision: the proportion of true positives among predicted positives;
• selection rate: the proportion of the total population classified as positive;
• calibration: Whether the same risk score corresponds to similar actual occurrence rates.

If the model decides who gets scarce aid, the harms of underreporting and overreporting are different; if used for punishment, the cost of false positives might be higher. Metric selection should be derived from the decision process and the pathways of harm.

2. Common Fairness Goals for Answering Different Questions

Demographic parity

All groups receive positive decisions at the same rate. It focuses on distribution outcomes but may overlook genuine needs or historical labeling differences.

Equal opportunity

The true positive rate is the same across groups. It focuses on whether qualified or needy individuals are equally identified.

Equalized odds

All groups have the same true positive rate and false positive rate. Stronger constraints may require adjusting the threshold or the model.

Calibration within groups

Predicted scores align closely with actual risks across different groups, making it suitable for risk communication, but if labels are biased, calibration will faithfully reproduce that bias.

When group baseline rates differ and predictions are imperfect, several fairness criteria often can't all be satisfied. Choosing isn't purely a technical optimization; it's a governance decision about which kinds of harm should be prioritized.

3. Minimum Table for Grouped Evaluation

import pandas as pd
from sklearn.metrics import confusion_matrix

def group_metrics(part):
    tn, fp, fn, tp = confusion_matrix(
        part["y_true"], part["y_pred"], labels=[0, 1]
    ).ravel()
    return pd.Series({
        "n": len(part),
        "selection_rate": (tp + fp) / max(len(part), 1),
        "tpr": tp / max(tp + fn, 1),
        "fpr": fp / max(fp + tn, 1),
        "precision": tp / max(tp + fp, 1),
    })

report = evaluations.groupby("region").apply(group_metrics)

For real projects, molecular, denominational, and uncertainty intervals must be provided. When the denominator is zero, it should not be faked as zero; instead, it should be marked as unestimable.

4. Cross-group and Small Sample

Looking at region, age, or device alone misses cross-group combinations like "North Region × Old Device × Night Shift." The finer the segmentation, the smaller the sample size, the wider the confidence intervals, and the greater the risk of multiple comparisons.

May include:

• Pre-identify high-risk cross-groups;
• Report raw counts and intervals, rather than rankings alone;
• Use layered models/partial aggregation to improve stability;
• When combining groups, preserve the damage semantics and avoid making rare groups disappear into "Other";
• Treat small samples as supplementary data and deploy signals with caution.

There is no universal "5% threshold." Thresholds should be established by considering effect size, statistical uncertainty, legal and policy considerations, severity of harm, and acceptable risk.

5. Labels Might Not Reflect the Full Truth

If the label is "formerly disciplined," it is influenced by behavior, inspection probability, reporting channels, and manager judgment. The model's accurate prediction of the label does not mean it is fair in reflecting actual behavior.

Censorship:

• Defines whether labels are consistent across groups;
• Who is more likely to be observed, reported, or confirmed;
• Are tag delays and tag omissions different;
• Does historical decision-making determine access to labeling opportunities;
• Are there alternative measures that are closer to the target construct?

Group metric differences may stem from variations in model, data, threshold, process, or real-world environment, requiring different remediation strategies.

6. Sensitive attributes can't be simply deleted

Deleting fields such as region or gender does not remove proxy information from other features and cannot enable group auditing. On the other hand, collecting sensitive attributes poses privacy and security risks.

Should be distinguished:

• Is it permitted for use in model training or decision-making;
• Used exclusively in a controlled environment for evaluation only;
• Who accesses, and at what granularity retains;
• How missing, self-reported, multiple identities, and changes are represented.

This decision requires input from the domain, law, privacy, and affected parties, and can't rely on "fairness through unawareness."

7. Remedies Must Target the Root Cause

• Inadequate coverage: improve data collection or narrow the scope of application;
• Measurement error: Fix equipment and labeling process;
• Thresholds lead to different levels of damage: revise decision rules when compliance and a legitimate reason exist;
• Model underfits a group: improve representations, loss, or model, but validate independently;
• Poor decision-making: disabling automation or altering resource rules;
• Feedback loop: Preserve measurement channels or exploration samples unaffected by the model.

Forcing one metric to be equal might worsen another metric or overall welfare. Every mitigation must be re-evaluated for performance, group impact, interpretability, and long-term behavior.

8. A Reviewable Fairness Report

Decision-use, Affected Parties, and Harm Priority
Group Definition, Data Sources, and Coverage Limitations
Sample sizes, labeling rates, prediction rates, and confusion matrices for each group
Poor effect/ratio and uncertainty interval
Cross-sectional and time-slice populations
Labels and Measurement Validity
The selected fair objectives and the rationale for not selecting others
Balancing trade-offs, residual risks, and responsibilities

Common Misconceptions

-Accuracy difference below a certain threshold is considered fair: there is no universal threshold for general disparity.

• Make the selection rate the same: It's just a possible target.
• Deleting sensitive attributes removes discrimination: proxy variables and processes remain.
• Small group metrics fluctuate, so ignore them: Uncertainty should prompt caution and additional evidence.

Practice

1. Select primary error indicators for resource support and disciplinary review, and explain the rationale.
2. Compute the confusion matrices, molecular denominators, and bootstrap intervals for the four groups.
3. Conduct cross-group analysis by device generation to uncover hidden issues in the overall mean.
4. Propose data, model, threshold, and process-layer fixes for a single difference, and compare their side effects.

Summary

A fairness assessment isn't about chasing a pretty overall score; it's about laying bare the decisions, mistakes, and uncertainties faced by different groups. Metrics are only governance tools when they're tied to harm and response.

The next lesson covers the power boundaries after launch: explanation, manual review, appeals, monitoring, and incident response must together form a truly executable control system.