10.2 Fairness Metrics and Group Evaluation: Which Errors Harm Whom?

The overall accuracy is 91%, but the North region only reaches 82%. This warrants investigation, yet we can't simply apply the rule that "a difference over 5% is unfair": sample size, label quality, baseline rates, and the consequences of error are all unknown. Fairness isn't a one-size-fits-all formula detached from context.

Learning Objectives

• Break down predictions, labels, and error rates by group;
• Distinguish demographic parity, equal opportunity, equalized odds, and calibration;
• Use confidence intervals and intersectional analysis to avoid misjudgments from small samples;
• Link metric differences to actionable mitigation measures.

1. What kind of errors does accuracy hide

For binary classification, first report the confusion matrix for each group:

• true positive rate: the proportion of actual positives correctly identified;
• false negative rate: the proportion of true positives that are incorrectly classified as negatives;
• false positive rate: the proportion of true negatives incorrectly identified as positives;
• precision: the proportion of true positives among predicted positives;
• selection rate: the proportion of the total population classified as positive;
• calibration: Whether the same risk score corresponds to similar actual occurrence rates.

If predictions allocate scarce support, false negatives and false positives cause different harms. For punitive decisions, false positives may be especially costly. Choose metrics from the decision and its paths to harm.

2. Common Fairness Goals for Answering Different Questions

Demographic parity

All groups receive positive decisions at the same rate. It focuses on distribution outcomes but may overlook genuine needs or historical labeling differences.

“Positive” means decision code one, not necessarily a benefit to the affected person. In disciplinary review, a positive prediction may trigger an investigation. Define label and action meanings first.

Equal opportunity

The true positive rate is the same across groups. It focuses on whether qualified or needy individuals are equally identified.

Equalized odds

All groups have the same true positive rate and false positive rate. Stronger constraints may require adjusting the threshold or the model.

Calibration within groups

Within each group, the same predicted score should correspond to the same observed event frequency: a calibrated score of 0.2 implies about 20% positives among sufficiently many comparable cases. Calibration can support risk communication, but biased labels remain biased calibration targets.

Fairness constraints can conflict, but distinguish score calibration, thresholded error rates, and precision rather than claiming that fairness is impossible. Hardt and colleagues define equalized odds and equal opportunity; Kleinberg and colleagues establish compatibility conditions for risk-score calibration and average-score balance in the positive and negative classes.

3. Minimum Table for Grouped Evaluation

import numpy as np
import pandas as pd
from scipy.stats import binomtest
from sklearn.metrics import confusion_matrix

def from_counts(region, tn, fp, fn, tp):
    pairs = [(0, 0)] * tn + [(0, 1)] * fp + [(1, 0)] * fn + [(1, 1)] * tp
    return pd.DataFrame(pairs, columns=["y_true", "y_pred"]).assign(region=region)

evaluations = pd.concat([
    from_counts("A", 64, 16, 4, 16),
    from_counts("B", 16, 4, 16, 64),
    from_counts("no_positives", 4, 1, 0, 0),
    from_counts("no_selected", 2, 0, 3, 0),
], ignore_index=True)

def ratio(numerator, denominator):
    return numerator / denominator if denominator else np.nan

def group_metrics(part):
    if not part[["y_true", "y_pred"]].isin([0, 1]).all().all():
        raise ValueError("binary nonmissing labels and predictions required")
    tn, fp, fn, tp = confusion_matrix(part["y_true"], part["y_pred"], labels=[0, 1]).ravel()
    positive_n = int(tp + fn)
    ci = binomtest(int(tp), positive_n).proportion_ci(method="wilson") if positive_n else None
    return {"n": len(part), "tn": tn, "fp": fp, "fn": fn, "tp": tp,
            "label_rate": ratio(tp + fn, len(part)),
            "selection_rate": ratio(tp + fp, len(part)),
            "tpr": ratio(tp, tp + fn), "fpr": ratio(fp, fp + tn),
            "precision": ratio(tp, tp + fp),
            "tpr_low": ci.low if ci else np.nan, "tpr_high": ci.high if ci else np.nan}

report = pd.DataFrame.from_dict({
    region: group_metrics(part)
    for region, part in evaluations.groupby("region", dropna=False)
}, orient="index")
print(report.loc[["A", "B"], ["tpr", "fpr", "selection_rate", "precision"]].round(6))
#    tpr  fpr  selection_rate  precision
# A  0.8  0.2            0.32   0.500000
# B  0.8  0.2            0.68   0.941176
assert pd.isna(report.loc["no_positives", "tpr"])
assert pd.isna(report.loc["no_selected", "precision"])

This example uses pandas, scikit-learn, and SciPy and preserves raw confusion counts. TPR is undefined without actual positives; precision is undefined without predicted positives. Replacing the denominator with one silently turns “not estimable” into “zero performance.” An unobserved group is not evidence that the group does not exist.

Both groups have TPR 0.8 and FPR 0.2, satisfying equalized odds in this sample. Yet base rates 0.2 and 0.8 produce selection rates 0.32 and 0.68 and precisions 0.5 and about 0.9412. Verify this using $P(\hat{Y}=1)=TPR\cdot \pi +FPR\cdot (1-\pi )$. A difference in thresholded precision is not itself a calibration test for the underlying continuous risk score.

4. Intersectional Groups and Small Samples

Looking at region, age, or device alone misses intersectional combinations like "North Region × Old Device × Night Shift." The finer the segmentation, the smaller the sample size, the wider the confidence intervals, and the greater the risk of multiple comparisons.

The TPR point estimates above match, but A has only 20 actual positives while B has 80, giving A a wider Wilson interval. These intervals assume independent binomial observations within each group. Repeated records per person or site sampling need dependence-aware inference. Empty denominators cannot be bootstrapped, and many subgroup intervals do not provide simultaneous coverage.

Possible approaches:

• Pre-identify high-risk intersectionals;
• Report raw counts and intervals, rather than rankings alone;
• Use hierarchical models/partial pooling to improve stability;
• When combining groups, preserve the meaning of the harm and avoid making rare groups disappear into "Other";
• Treat small samples as a signal to collect more evidence and deploy cautiously.

There is no universal "5% threshold." Thresholds should be established by considering effect size, statistical uncertainty, legal and policy considerations, severity of harm, and acceptable risk.

5. Labels Might Not Reflect the Full Truth

If the label is "formerly disciplined," it is influenced by behavior, inspection probability, reporting channels, and manager judgment. The model's accurate prediction of the label does not mean it is fair in reflecting actual behavior.

Review:

• Defines whether labels are consistent across groups;
• Who is more likely to be observed, reported, or confirmed;
• Do label delays and missingness differ;
• Does historical decision-making determine access to labeling opportunities;
• Are there alternative measures that are closer to the target construct?

Group metric differences may stem from variations in model, data, threshold, process, or real-world environment, requiring different remediation strategies.

6. Sensitive attributes can't be simply deleted

Deleting fields such as region or gender does not remove proxy information from other features and cannot enable group auditing. On the other hand, collecting sensitive attributes poses privacy and security risks.

Should be distinguished:

• Is it permitted for use in model training or decision-making;
• Used exclusively in a controlled environment for evaluation only;
• Who accesses, and at what granularity retains;
• How missing, self-reported, multiple identities, and changes are represented.

This decision requires input from the domain, law, privacy, and affected parties, and can't rely on "fairness through unawareness."

7. Remedies Must Target the Root Cause

• Inadequate coverage: improve data collection or narrow the scope of application;
• Measurement error: Fix equipment and labeling process;
• Thresholds lead to different levels of damage: revise decision rules when compliance and a legitimate reason exist;
• Model underfits a group: improve representations, loss, or model, but validate independently;
• Poor decision-making: disabling automation or altering resource rules;
• Feedback loop: Preserve measurement channels or exploration samples unaffected by the model.

Forcing one metric to be equal might worsen another metric or overall welfare. Every mitigation must be re-evaluated for performance, group impact, interpretability, and long-term behavior.

8. A Reviewable Fairness Report

Decision-use, Affected Parties, and Harm Priority
Group Definition, Data Sources, and Coverage Limitations
Sample sizes, labeling rates, prediction rates, and confusion matrices for each group
Effect differences/ratios and uncertainty intervals
Intersectional groups and time slices
Labels and Measurement Validity
The selected fair objectives and the rationale for not selecting others
Before-after mitigation trade-offs, residual risks, and owners

Common Misconceptions

• An accuracy difference below a threshold means the system is fair: There is no universal threshold for disparity.
• Make the selection rate the same: It's just a possible target.
• Deleting sensitive attributes removes discrimination: proxy variables and processes remain.
• Small group metrics fluctuate, so ignore them: Uncertainty should prompt caution and additional evidence.

Practice

1. Select primary error indicators for resource support and disciplinary review, and explain the rationale.
2. Compute the confusion matrices, numerators and denominators, and bootstrap intervals for the four groups.
3. Conduct intersectional analysis by device generation to uncover hidden issues in the overall mean.
4. Propose data, model, threshold, and process-layer fixes for a single difference, and compare their side effects.

Summary

A fairness assessment isn't about chasing a pretty overall score; it's about laying bare the decisions, mistakes, and uncertainties faced by different groups. Metrics are only governance tools when they're tied to harm and response.

The next lesson covers the power boundaries after launch: explanation, manual review, appeals, monitoring, and incident response must together form a truly executable control system.