跳转至

MLASTG-TEST-MODEL-001: Adversarial Robustness & Evasion Testing

Control Reference

  • MLASVS-MODEL-001: Adversarial Robustness Testing
  • MLASVS-MODEL-002: Input Perturbation Limits
  • MLASVS-MODEL-003: Model Input Validation
  • MLASVS-MODEL-010: Anomalous Input Detection
  • MLASVS-MODEL-012: Resource Limits on Inference
  • MLASVS-MODEL-016: Certified Adversarial Robustness - L2
  • MLASVS-MODEL-017: Robustness Certification - L2
  • MLASVS-MODEL-024: Adversarial Training Validation - L2
  • MLASVS-MODEL-025: Feature Squeezing Validation - L2
  • MLASVS-MODEL-026: Model Ensemble Diversity - L2
  • MLASVS-MODEL-027: Certified Defense Mechanisms - L2
  • MLASVS-MODEL-028: Red Team Exercise Completion - L2
  • MLASVS-MODEL-029: Continuous Adversarial Retesting - L2

Severity

High (L1) / Critical (L2)

Prerequisites

Requirement Details
Tools IBM ART (pip install adversarial-robustness-toolbox)
Model Access White-box (architecture + weights) or black-box (API access)
Test Data Minimum 1000 representative test samples
Compute GPU recommended for iterative attacks (PGD, CW)

Step-by-Step Procedure

Step 1: Establish Baseline Performance

  1. Load the target model and test dataset.
  2. Evaluate clean (unperturbed) accuracy.
  3. Record baseline metrics using standard metrics.
  4. Pass condition: Baseline accuracy meets business requirements (e.g., >90%).
  5. Fail condition: Baseline accuracy is too low to perform meaningful robustness testing.

Step 2: Execute White-Box Evasion Attacks (L1)

  1. Generate adversarial examples using Fast Gradient Sign Method (FGSM) at moderate perturbation budgets (e.g., ε = 0.1).
  2. Generate adversarial examples using Projected Gradient Descent (PGD).
  3. Evaluate model accuracy on the generated adversarial examples.
  4. Pass condition: Accuracy drop is ≤ 25% under moderate FGSM attack.
  5. Fail condition: Accuracy drop is > 25% under moderate FGSM attack.

Step 3: Execute Black-Box Evasion Attacks (L1)

  1. Generate adversarial examples using HopSkipJump attack (decision-based).
  2. Measure the degradation in model accuracy.
  3. Pass condition: Accuracy drop is ≤ 25% under HopSkipJump attack.
  4. Fail condition: Accuracy drop is > 25% under HopSkipJump attack.

Step 4: Analyze Robustness Metrics

  1. Summarize accuracy under each attack in a results table.
  2. Verify if the degradation is within acceptable limits for the given severity level.
  3. Pass condition: All robustness metrics fall within acceptable thresholds for the tested level.
  4. Fail condition: One or more robustness metrics exceed the acceptable degradation thresholds.

Step 5: Test Input Validation (L1)

  1. Send inputs outside the expected range (very large/small values).
  2. Send inputs with incorrect dimensions.
  3. Send inputs with NaN/Inf values.
  4. Pass condition: Invalid inputs are rejected with appropriate errors and not processed.
  5. Fail condition: Invalid inputs are processed by the model without errors, leading to unexpected outputs or crashes.

Step 6: Execute Certified Robustness Evaluation (L2)

  1. Implement randomized smoothing certification.
  2. Document certified accuracy at different radii.
  3. Pass condition: Certified accuracy ≥ 80% at radius r ≥ 0.1 (L2 norm).
  4. Fail condition: Certified accuracy is < 80% at radius r ≥ 0.1 (L2 norm).

Step 7: Validate Adversarial Training (L2)

  1. Verify the training pipeline includes adversarial training (e.g., PGD-AT).
  2. Compare robust accuracy of the adversarially trained model against the standard trained baseline under the same attack budget.
  3. Test against attack types not used during adversarial training to assess generalization.
  4. Pass condition: Adversarially trained model achieves >15% higher robust accuracy than the standard trained model under the same attack budget.
  5. Fail condition: The model does not demonstrate a significant (>15%) improvement in robust accuracy, or generalizes poorly to unseen attack types.

Step 8: Test Input Detection Capabilities (L2)

  1. Deploy a feature squeezing defense or similar detection mechanism.
  2. Test detection rate of adversarial inputs.
  3. Pass condition: Detection rate > 80% with false positive rate < 5%.
  4. Fail condition: Detection rate is ≤ 80% or false positive rate is ≥ 5%.

Step 9: Review Red Team Reports (L2)

  1. Verify red team exercises were conducted within the last quarter.
  2. Review findings for evasion-related vulnerabilities.
  3. Confirm findings are tracked to remediation.
  4. Pass condition: Red team exercises are conducted at least quarterly with all evasion findings tracked and remediated.
  5. Fail condition: Red team exercises are missing, outdated, or findings are unaddressed.

Expected Result

Level Expected Outcome
L1 Accuracy drop ≤ 25% under FGSM (ε=0.1) and PGD (ε=0.3, 40 steps); input validation enforces bounds.
L2 Certified robust accuracy ≥ 80% at r ≥ 0.1; adversarial training validated; quarterly red team exercises; continuous testing in CI/CD.

Evidence Requirements

  • Baseline clean accuracy
  • Accuracy under FGSM, PGD, and one black-box attack
  • Input validation test results
  • (L2) Certified robustness evaluation results
  • (L2) Adversarial training comparison
  • (L2) Feature squeezing detection results
  • (L2) Red team exercise reports
  • (L2) CI/CD robustness test integration evidence

Remediation Guidance

  • Implement adversarial training with PGD-based examples
  • Apply feature squeezing preprocessing
  • Increase model capacity/resilience
  • Implement ensemble methods for diverse defenses
  • Consider certified defenses (randomized smoothing)

References

  • MITRE ATLAS: AML.T0010 - Adversarial Examples (Evasion)
  • MITRE ATLAS: AML.T0029 - Craft Adversarial Data
  • MITRE ATLAS: AML.T0043 - Discover Model Ontology
  • MLASWE: MLASWE-0001 (Adversarial Perturbation)