MLASTG Testing Methodology¶

1. Overview¶

This document defines the methodology for conducting a structured security assessment of machine learning systems using the MLASVS (ML Application Security Verification Standard) and MLASWE (ML Application Security Weakness Enumeration).

The methodology follows a phased approach adapted from the Penetration Testing Execution Standard (PTES) and OWASP Testing Guide, and extended to address the unique attack surface of ML systems.

2. The ML Security Assessment Lifecycle¶

Phase 1: Planning & Scoping
    │
    ▼
Phase 2: Intelligence Gathering & Threat Modeling
    │
    ▼
Phase 3: Control Verification (MLASTG Test Cases)
    ├── Data Security Tests
    ├── Model Security Tests
    ├── LLM Security Tests
    ├── Supply Chain Tests
    ├── Pipeline Tests
    ├── Infrastructure Tests
    └── Governance Tests
    │
    ▼
Phase 4: Exploitation & Validation
    │
    ▼
Phase 5: Reporting & Remediation

3. Phase 1: Planning & Scoping¶

3.1 Define Scope¶

• Identify ML components: Models, training pipelines, inference endpoints, feature stores, model registries
• Determine ML system type: Classification, regression, NLP, computer vision, LLM/GenAI, recommendation
• Map data flows: Training data sources → preprocessing → training → evaluation → deployment → inference
• Identify dependencies: Pre-trained models, third-party APIs, external datasets, ML libraries

3.2 Select Security Level¶

3.3 Establish Rules of Engagement¶

• Define testing window and notification requirements
• Determine if adversarial testing (evasion, extraction) is in scope
• Establish model availability requirements (avoid DoS to production models)
• Define data handling for any training data or model artifacts accessed during testing

4. Phase 2: Intelligence Gathering & Threat Modeling¶

4.1 Architecture Discovery¶

• Document ML system architecture (diagrams, data flows, component interactions)
• Identify all ML artifacts (datasets, models, pipelines, serving infrastructure)
• Map trust boundaries between components

4.2 Threat Modeling Using MITRE ATLAS¶

Map the system architecture to MITRE ATLAS tactics:

4.3 Tool Setup¶

5. Phase 3: Control Verification¶

5.1 Test Execution Flow¶

For each applicable MLASVS control:

1. Identify the control from the applicable category
2. Reference the corresponding MLASTG-TEST-XXXX test case
3. Execute the test procedure
4. Document results with evidence
5. Classify any findings using MLASWE identifiers

5.2 Test Categories¶

5.3 Test Procedure Template¶

Each test case follows this structure:

## MLASTG-TEST-XXXX: [Test Title]

### Control Reference
MLASVS-{CATEGORY}-{NUMBER}: {Control Description}

### Severity
Critical / High / Medium / Low / Informational

### Prerequisites
- Access requirements
- Tools required
- Model/data access needed

### Step-by-Step Procedure
1. [Step 1]
2. [Step 2]
3. [Step 3]
   ...

### Expected Result
[What should happen if the control is properly implemented]

### Evidence Requirements
- [What to capture as proof of testing]

### Remediation Guidance
[How to fix if the test fails]

### References
- MITRE ATLAS: AML.TXXXX
- MLASWE: MLASWE-XXXX

6. Phase 4: Exploitation & Validation¶

6.1 Adversarial Testing¶

For L2 assessments, adversarial exploitation is in scope:

6.2 Validation Criteria¶

7. Phase 5: Reporting¶

7.1 Report Structure¶

Executive Summary
├── Assessment scope and methodology
├── Risk overview by category
├── Key findings and critical risks
└── Overall security posture

Technical Findings
├── By MLASVS Category
│   ├── Control Reference
│   ├── MLASWE Reference
│   ├── Test Procedure Executed
│   ├── Evidence
│   ├── Risk Rating
│   └── Remediation Recommendation
└── Adversarial Testing Results
    ├── Attack type
    ├── Success rate
    ├── Affected controls
    └── Mitigation effectiveness

Compliance Mapping
├── MLASVS Coverage (%)
├── Controls by level (L1/L2)
├── Industry framework alignment
└── Gap analysis

Appendices
├── Test execution logs
├── Tool outputs and screenshots
├── Model architecture diagrams
└── ML-SBOM analysis

7.2 Risk Ratings¶

7.3 Remediation Timing¶