MLASTG-TEST-MODEL-003: Membership Inference Testing¶

Control Reference¶

Controls Tested: MLASVS-MODEL-009 (Inference Logging), MLASVS-MODEL-011 (Output Sanitization), MLASVS-MODEL-013 (Model Behavior Monitoring), MLASVS-MODEL-019 (Differential Privacy in Model — L2), MLASVS-MODEL-020 (Membership Inference Prevention — L2)

Severity¶

Medium (L1) / High (L2)

Overview¶

Membership inference attacks determine whether a specific data record was part of a model's training dataset by exploiting the model's tendency to be more confident on training data. A successful attack can expose sensitive training data membership and violates privacy guarantees.

Prerequisites¶

Step-by-Step Procedure¶

Step 1: Shadow Model Membership Inference Attack¶

1. Split known training data and non-training data into balanced evaluation sets
2. Train shadow models on data resembling the target's training distribution
3. Build an attack classifier using the shadow model outputs as features:

   import numpy as np
   from art.attacks.inference.membership_inference import MembershipInferenceBlackBox
   from art.estimators.classification import PyTorchClassifier
   
   # Wrap target model
   target = PyTorchClassifier(
       model=target_model, loss=criterion,
       input_shape=input_shape, nb_classes=num_classes
   )
   
   # Run black-box membership inference attack
   attack = MembershipInferenceBlackBox(target, attack_model_type="rf")
   attack.fit(x_train, y_train, x_test, y_test)
   
   # Infer membership on evaluation set
   member_predictions = attack.infer(x_train[:100], y_train[:100])
   non_member_predictions = attack.infer(x_test[:100], y_test[:100])
   
   # Compute attack accuracy
   member_labels = np.ones(100)      # ground truth: member
   non_member_labels = np.zeros(100) # ground truth: non-member
   all_preds  = np.concatenate([member_predictions, non_member_predictions])
   all_labels = np.concatenate([member_labels, non_member_labels])
   attack_accuracy = np.mean(all_preds == all_labels)
   print(f"Attack accuracy: {attack_accuracy:.4f} (random baseline: 0.50)")
   

4. Pass if: Attack accuracy is < 0.60 (within 10% above random guessing baseline of 0.50)
5. Fail if: Attack accuracy ≥ 0.60 — the model leaks meaningful membership information

Step 2: Verify Inference Logging¶

1. Review inference logs for completeness and format
2. Verify logs capture: timestamp, request identifier, model version, input shape (not raw input), output label (not raw probabilities)
3. Verify logs do NOT contain raw PII, training data records, or full probability vectors
4. Pass if: Inferences are logged with anonymized identifiers; no PII or training data appears in logs

Step 3: Verify Output Sanitization¶

1. Query model with inputs known to be in the training set
2. Compare confidence levels against inputs known to be out of distribution
3. Verify confidence scores do not systematically differ in a way that enables membership inference without an attack model
4. Pass if: Average confidence gap between training and non-training samples is < 5%

Step 4: Verify Differential Privacy (L2)¶

1. Request DP audit documentation from the model development team
2. Verify that the privacy budget ε (epsilon) is documented and tracked
3. Verify training used a DP-SGD or equivalent mechanism (e.g., Opacus, TF Privacy)
4. Pass if: ε ≤ 10 for standard sensitivity scenarios. For high-sensitivity data (healthcare, finance), ε ≤ 2 is recommended. Document and justify any value above ε = 2.

Note: ε values are context-dependent starting guidance. Stricter budgets should be applied based on data sensitivity classification. An ε = 10 with no justification is not acceptable for sensitive data.

Step 5: Verify Membership Inference Prevention Controls¶

1. Verify at least one active prevention control is deployed:
2. Output precision limiting (confidence scores truncated to ≤ 3 decimal places)
3. Prediction perturbation / randomized response
4. Differential privacy training (documented ε)
5. Label-only API (no confidence scores exposed)
6. Pass if: At least one prevention control is implemented and documented

Expected Result¶

Evidence Requirements¶

• Membership inference attack accuracy results
• Inference log review results (no PII confirmed)
• Output sanitization test results (confidence gap measurement)
• (L2) Differential privacy documentation with ε value
• (L2) DP training mechanism evidence (Opacus config, TF Privacy logs, etc.)
• Active prevention control documentation

Remediation Guidance¶

If attack accuracy is high: 1. Implement differential privacy during training (Opacus for PyTorch, TF Privacy for TensorFlow) 2. Limit confidence score precision (truncate to ≤ 3 decimal places) 3. Use label-only prediction API (return predicted class only, no probabilities) 4. Apply L2 regularization and early stopping to reduce overfitting (high overfitting correlates with high MI attack success)

If logs contain sensitive data: 1. Remove raw input data from inference logs immediately 2. Implement structured logging with a defined schema that excludes raw inputs 3. Conduct a log audit and purge any non-compliant log entries

References¶

• MITRE ATLAS:
• AML.T0018 — Backdoor ML Model / membership inference context
• AML.T0041 — ML Model Inference API Access
• MLASWE: MLASWE-0004 (Training Data Leakage), MLASWE-0005 (Privacy Violation through Inference)
• NIST AI RMF: MANAGE 2.4, MEASURE 2.7
• Academic: Shokri et al. (2017) "Membership Inference Attacks Against Machine Learning Models"