# Chapter 70 — Secure AI Application Security Testing & Verification

![Post cover](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/jlsqnyoejsq1u5i3z5nl.png align="center")

  

## 70.1 Introduction

Security testing is the verification layer of a secure AI system.

A security architecture may contain authentication, authorization, encryption, policy enforcement, sandboxing, logging, monitoring, and isolation. However, none of these controls should be considered trustworthy merely because they exist in source code or configuration.

The system must be tested continuously.

A useful security-assurance model is:

**Design → Implement → Test → Observe → Fix → Retest → Continuously Monitor**

For AI applications, conventional application-security testing is necessary but insufficient. AI systems introduce additional attack surfaces involving:

*   prompts
    
*   model inputs
    
*   model outputs
    
*   retrieval systems
    
*   embeddings
    
*   tool calls
    
*   agents
    
*   memory
    
*   multimodal inputs
    
*   model files
    
*   third-party providers
    
*   plugins
    
*   generated content
    
*   autonomous workflows
    

Therefore, AI security testing should combine traditional application security with AI-specific evaluation.

* * *

# 70.2 Security Testing Objectives

The primary objectives are:

1.  Discover vulnerabilities before deployment.
    
2.  Verify security controls actually enforce intended policies.
    
3.  Detect regressions after code or model changes.
    
4.  Validate tenant isolation.
    
5.  Test authentication and authorization boundaries.
    
6.  Detect unsafe input handling.
    
7.  Test resistance to prompt injection.
    
8.  Test AI output controls.
    
9.  Validate tool permissions.
    
10.  Test file and media processing.
     
11.  Verify secrets are not exposed.
     
12.  Validate infrastructure configuration.
     
13.  Test dependencies and containers.
     
14.  Verify logging and detection capabilities.
     
15.  Measure recovery after security failures.
     

Security testing should therefore answer two different questions:

**“Can an attacker break this?”**

and

**“Will the system detect and contain the attack if it happens?”**

* * *

# 70.3 Security Testing Layers

A mature AI platform should test multiple layers.

| Layer | Primary Testing |
| --- | --- |
| Source code | SAST |
| Dependencies | SCA |
| Secrets | Secret scanning |
| Infrastructure | IaC scanning |
| Containers | Container scanning |
| APIs | API security testing |
| Web application | DAST |
| AI prompts | Prompt-security testing |
| Models | AI robustness testing |
| RAG | Retrieval security testing |
| Agents | Agent/tool security testing |
| Media pipeline | File/codec/resource testing |
| Cloud | Configuration/security testing |
| Runtime | Behavioral monitoring |
| Human processes | Security procedures |
| Production | Continuous assurance |

No single scanner can provide complete coverage.

* * *

# 70.4 Static Application Security Testing

Static Application Security Testing (SAST) analyzes source code without executing the application.

Typical findings include:

*   injection vulnerabilities
    
*   insecure authentication logic
    
*   authorization mistakes
    
*   unsafe cryptographic usage
    
*   insecure deserialization
    
*   path traversal
    
*   command execution risks
    
*   dangerous API usage
    
*   hardcoded secrets
    
*   insecure redirects
    
*   unsafe file operations
    

For an AI application, SAST should additionally inspect:

*   prompt construction
    
*   tool authorization
    
*   model-provider calls
    
*   system prompt handling
    
*   retrieval authorization
    
*   memory access
    
*   tenant identifiers
    
*   output handling
    
*   agent state transitions
    

For example, an AI service should not simply do:

```text
user request
    ↓
LLM
    ↓
tool execution
```

The security architecture should instead resemble:

```text
user request
    ↓
authentication
    ↓
authorization
    ↓
input validation
    ↓
policy evaluation
    ↓
LLM
    ↓
tool-request validation
    ↓
permission check
    ↓
sandboxed execution
    ↓
result validation
    ↓
response policy
    ↓
user
```

SAST can help verify that these security boundaries exist in code.

* * *

# 70.5 Software Composition Analysis

Modern applications depend on hundreds or thousands of external packages.

Software Composition Analysis (SCA) identifies:

*   vulnerable dependencies
    
*   outdated libraries
    
*   transitive dependencies
    
*   abandoned packages
    
*   license risks
    
*   known vulnerable versions
    
*   dependency relationships
    

For a JavaScript/TypeScript AI application, dependency testing should cover:

```text
package.json
package-lock.json
npm dependencies
transitive dependencies
native libraries
container packages
AI SDKs
database drivers
media libraries
authentication libraries
```

Dependency versions should be reproducible.

A production build should preferably use a lockfile and controlled dependency-resolution process.

* * *

# 70.6 Secret Scanning

Secrets are among the most dangerous accidental security failures.

Testing should search for:

*   API keys
    
*   cloud credentials
    
*   database passwords
    
*   signing keys
    
*   private keys
    
*   OAuth secrets
    
*   webhook secrets
    
*   service-account credentials
    
*   encryption keys
    
*   provider tokens
    

Scanning should occur at multiple stages:

```text
Developer workstation
        ↓
Pre-commit
        ↓
Pull request
        ↓
CI pipeline
        ↓
Container/image inspection
        ↓
Deployment validation
        ↓
Repository history
```

A secret scanner should not be the only control.

If a real secret is discovered, the correct response is generally:

```text
Detect
 ↓
Revoke
 ↓
Rotate
 ↓
Investigate usage
 ↓
Remove exposure
 ↓
Retest
```

Simply deleting the secret from the latest commit may not be sufficient because it may remain in repository history, build artifacts, logs, caches, or backups.

* * *

# 70.7 Infrastructure-as-Code Security Testing

Infrastructure-as-Code (IaC) defines infrastructure through configuration.

Examples include:

*   Terraform
    
*   Kubernetes manifests
    
*   Helm charts
    
*   cloud deployment templates
    
*   network policies
    
*   IAM policies
    

IaC testing should identify configurations such as:

*   public storage buckets
    
*   unrestricted network access
    
*   excessive IAM permissions
    
*   insecure databases
    
*   disabled encryption
    
*   exposed administrative interfaces
    
*   privileged containers
    
*   missing network policies
    
*   insecure Kubernetes settings
    

A secure deployment pipeline can enforce:

```text
IaC change
   ↓
syntax validation
   ↓
policy validation
   ↓
security scanning
   ↓
review
   ↓
test environment
   ↓
security verification
   ↓
production
```

* * *

# 70.8 Container Security Testing

AI workloads frequently run inside containers.

Container security testing should examine:

*   base image vulnerabilities
    
*   operating-system packages
    
*   application dependencies
    
*   unnecessary utilities
    
*   root execution
    
*   excessive Linux capabilities
    
*   exposed ports
    
*   writable filesystem requirements
    
*   embedded secrets
    
*   suspicious binaries
    

A preferred container model is:

```text
Minimal base image
        ↓
Pinned dependencies
        ↓
Application build
        ↓
Security scan
        ↓
SBOM generation
        ↓
Signed image
        ↓
Controlled registry
        ↓
Runtime verification
```

Containers should be treated as security boundaries, but not as the only security boundary.

* * *

# 70.9 Dynamic Application Security Testing

Dynamic Application Security Testing (DAST) evaluates a running application.

It can identify weaknesses such as:

*   authentication failures
    
*   authorization problems
    
*   injection
    
*   insecure headers
    
*   exposed endpoints
    
*   session weaknesses
    
*   unexpected error disclosure
    
*   insecure file handling
    

DAST should operate against a dedicated test environment.

Testing production systems without appropriate authorization can cause outages or unintended data modification.

* * *

# 70.10 API Security Testing

AI platforms often expose APIs such as:

```text
POST /api/auth/login
POST /api/generate
POST /api/chat
POST /api/upload
POST /api/search
POST /api/agent/run
POST /api/workflows
GET  /api/projects
GET  /api/history
```

Each endpoint should be tested for:

*   authentication
    
*   authorization
    
*   object-level access control
    
*   rate limiting
    
*   input validation
    
*   output filtering
    
*   schema validation
    
*   replay resistance
    
*   request size limits
    
*   timeout handling
    
*   error handling
    
*   tenant isolation
    

A particularly important test is:

```text
User A owns object A.

Can User B access object A?
```

The expected answer must always be:

**No.**

This should be tested automatically.

* * *

# 70.11 Authorization Regression Testing

Authorization vulnerabilities are often caused by small changes.

For example:

```text
Original:

GET /projects/{projectId}

Authorization:
user owns project
```

Later, a developer may accidentally change the implementation to:

```text
SELECT project WHERE id = projectId
```

without verifying ownership.

The application still works functionally, but the security boundary has disappeared.

Therefore authorization tests should be permanent regression tests.

Example:

```text
create user A
create user B

create project owned by A

authenticate as B

request project A

expected:
403 Forbidden
```

The test should fail the build if User B receives the protected resource.

* * *

# 70.12 Fuzz Testing

Fuzzing supplies unexpected or malformed inputs to software.

Useful fuzz targets include:

*   JSON parsers
    
*   API endpoints
    
*   upload handlers
    
*   image processors
    
*   video processors
    
*   audio processors
    
*   document parsers
    
*   metadata parsers
    
*   prompt parsers
    
*   workflow definitions
    
*   tool arguments
    

Inputs can vary in:

*   length
    
*   encoding
    
*   structure
    
*   nesting
    
*   missing fields
    
*   duplicate fields
    
*   unusual Unicode
    
*   unexpected types
    
*   boundary values
    

For AI applications, fuzzing should also cover multimodal input boundaries.

* * *

# 70.13 File Upload Security Testing

File uploads require dedicated testing.

Potential attack classes include:

*   malicious filenames
    
*   path traversal
    
*   MIME-type spoofing
    
*   oversized files
    
*   decompression bombs
    
*   malformed images
    
*   malformed documents
    
*   malformed audio
    
*   malformed video
    
*   parser vulnerabilities
    
*   metadata abuse
    

The secure architecture should be:

```text
Upload
  ↓
Authentication
  ↓
Authorization
  ↓
Size limit
  ↓
Content-type validation
  ↓
Magic-byte validation
  ↓
Quarantine
  ↓
Malware scanning
  ↓
Sandboxed parsing
  ↓
Transformation
  ↓
Output validation
  ↓
Trusted storage
```

The original upload should not automatically become a trusted asset.

* * *

# 70.14 Prompt Injection Testing

AI systems must be tested against prompt injection.

A simplified attack structure is:

```text
Trusted instruction
       +
untrusted content
       ↓
model interpretation
       ↓
instruction conflict
```

Testing should include:

*   direct prompt injection
    
*   indirect prompt injection
    
*   retrieved-document injection
    
*   webpage injection
    
*   malicious metadata
    
*   tool-result injection
    
*   memory poisoning
    
*   cross-modal injection
    

The security objective is not merely:

**“Does the model refuse?”**

It is also:

**“Can the injected content cause a protected action?”**

That distinction is critical.

* * *

# 70.15 RAG Security Testing

Retrieval-Augmented Generation systems create additional security boundaries.

Tests should verify:

```text
User A
 ↓
authorized documents only
```

and prevent:

```text
User A
 ↓
retrieval
 ↓
User B's documents
```

Testing should cover:

*   unauthorized retrieval
    
*   metadata leakage
    
*   cross-tenant retrieval
    
*   poisoned documents
    
*   malicious instructions in documents
    
*   stale authorization metadata
    
*   deleted-document retrieval
    
*   embedding-store isolation
    

A document should not become accessible merely because it was successfully embedded.

Authorization must remain part of retrieval.

* * *

# 70.16 AI Output Security Testing

AI outputs should not automatically be trusted.

Outputs may contain:

*   unsafe HTML
    
*   malicious URLs
    
*   executable code
    
*   shell commands
    
*   sensitive information
    
*   hallucinated credentials
    
*   inappropriate tool arguments
    
*   unsafe structured data
    

Therefore:

```text
Model output
    ↓
schema validation
    ↓
policy validation
    ↓
security filtering
    ↓
context-specific encoding
    ↓
application
```

For example, generated HTML should be treated as untrusted input.

* * *

# 70.17 AI Red-Team Testing

AI red teaming evaluates whether an AI system behaves safely under adversarial conditions.

A red-team program can test:

### Model behavior

*   harmful requests
    
*   policy circumvention
    
*   instruction conflicts
    
*   jailbreak attempts
    
*   multilingual attacks
    
*   adversarial formatting
    

### System behavior

*   prompt injection
    
*   data leakage
    
*   unauthorized tool use
    
*   memory abuse
    
*   retrieval manipulation
    

### Infrastructure behavior

*   rate-limit bypass
    
*   resource exhaustion
    
*   malformed requests
    
*   API abuse
    

### Agent behavior

*   unauthorized actions
    
*   excessive tool use
    
*   privilege escalation
    
*   unsafe planning
    
*   failure to request approval
    

AI red teaming should be repeatable rather than purely manual.

* * *

# 70.18 Multilingual Security Testing

A globally accessible AI application cannot assume English-only security behavior.

Security tests should include languages relevant to the platform.

Important categories include:

*   direct translations
    
*   mixed-language prompts
    
*   code-switching
    
*   transliteration
    
*   Unicode variations
    
*   regional terminology
    
*   indirect instructions
    
*   multilingual prompt injection
    

A system that blocks an unsafe request in English but executes the equivalent request in another language has a security inconsistency.

Therefore security policies should be tested across supported languages.

* * *

# 70.19 Multimodal Security Testing

AI media systems may process:

*   text
    
*   images
    
*   audio
    
*   video
    
*   documents
    

Each modality introduces different attack surfaces.

Example:

```text
Image
 ↓
OCR
 ↓
Extracted text
 ↓
LLM
 ↓
Tool decision
```

An image can therefore become an indirect source of instructions.

Similarly:

```text
PDF
 ↓
text extraction
 ↓
retrieval
 ↓
LLM
```

The extracted content must remain untrusted.

Multimodal testing should therefore verify that untrusted content cannot override system policy.

* * *

# 70.20 Agent Security Testing

Agents require special testing because they may take actions.

Testing should verify:

```text
Agent request
 ↓
identity
 ↓
permission
 ↓
policy
 ↓
tool authorization
 ↓
action
```

Important tests include:

*   unauthorized tool invocation
    
*   tool argument manipulation
    
*   privilege escalation
    
*   cross-user access
    
*   excessive action chains
    
*   missing human approval
    
*   unsafe retry behavior
    
*   malicious tool output
    
*   prompt injection through tools
    
*   state corruption
    

An agent should not be trusted merely because the model generated the action.

* * *

# 70.21 Security Unit Tests

Security logic should have ordinary automated tests.

Example categories:

```text
Authentication tests
Authorization tests
Tenant-isolation tests
Input-validation tests
Rate-limit tests
File-upload tests
Encryption tests
Audit-log tests
Policy-engine tests
Tool-permission tests
```

Security tests should execute automatically in CI.

* * *

# 70.22 Integration Security Testing

Unit tests verify individual functions.

Integration tests verify interactions.

For example:

```text
Authentication
      ↓
Authorization
      ↓
API
      ↓
Database
      ↓
Object storage
      ↓
AI service
```

A vulnerability can exist between components even if every individual component passes its own tests.

Integration tests should therefore verify end-to-end security properties.

* * *

# 70.23 End-to-End Security Testing

End-to-end tests simulate complete user workflows.

Example:

```text
Register
 ↓
Login
 ↓
Create project
 ↓
Upload file
 ↓
Generate AI content
 ↓
Save result
 ↓
Retrieve history
 ↓
Delete project
```

Security expectations should be checked at every stage.

For example:

*   deleted data cannot be retrieved
    
*   another tenant cannot access the project
    
*   unauthorized tools cannot execute
    
*   expired sessions cannot access protected APIs
    

* * *

# 70.24 Threat Modeling as a Testing Input

Threat modeling should directly influence security tests.

A practical loop is:

```text
Threat model
    ↓
Identify security properties
    ↓
Create test cases
    ↓
Automate critical tests
    ↓
Execute
    ↓
Measure failures
    ↓
Update threat model
```

This creates traceability between:

**Threat → Control → Test → Evidence**

That is much stronger than performing random security scans.

* * *

# 70.25 Security Test Cases as Security Requirements

Security requirements should be expressed as testable statements.

Example:

```text
Requirement:
Users must only retrieve objects they are authorized to access.

Test:
Authenticated User B requests User A's object.

Expected:
Access denied.
```

Another:

```text
Requirement:
Expired authentication sessions cannot access protected resources.

Test:
Use expired session.

Expected:
Authentication failure.
```

Another:

```text
Requirement:
Agent tools require explicit authorization.

Test:
Attempt unauthorized tool invocation.

Expected:
Tool execution denied.
```

This converts security policy into measurable behavior.

* * *

# 70.26 Continuous Security Regression Testing

Security testing should continue after launch.

A code change can accidentally break:

*   authentication
    
*   authorization
    
*   filtering
    
*   rate limiting
    
*   encryption
    
*   tenant isolation
    
*   logging
    
*   policy enforcement
    

Therefore critical security tests should run on every relevant change.

A simplified CI pipeline:

```text
Commit
 ↓
SAST
 ↓
Secret scan
 ↓
SCA
 ↓
Unit tests
 ↓
Security tests
 ↓
IaC scan
 ↓
Container scan
 ↓
Build
 ↓
Integration tests
 ↓
DAST
 ↓
AI security tests
 ↓
Approval
 ↓
Deployment
```

* * *

# 70.27 Security Gates

Not every finding should block deployment.

Organizations should define severity-based gates.

Example:

| Finding | Example Action |
| --- | --- |
| Critical vulnerability | Block |
| High exploitable vulnerability | Usually block |
| Exposed secret | Block |
| Broken tenant isolation | Block |
| Authentication bypass | Block |
| Medium issue | Review/remediation |
| Low informational issue | Track |

The exact policy should be adapted to the application's risk model.

* * *

# 70.28 False Positives and False Negatives

Security scanners are imperfect.

A **false positive** occurs when a tool reports a vulnerability that is not actually exploitable.

A **false negative** occurs when a real vulnerability is missed.

Therefore:

```text
Scanner
 +
Human review
 +
Automated tests
 +
Runtime monitoring
 +
Red teaming
```

should work together.

Security should never depend on one scanner.

* * *

# 70.29 Penetration Testing

Penetration testing provides deeper adversarial assessment.

For an AI platform, authorized testing can cover:

*   web application
    
*   APIs
    
*   authentication
    
*   authorization
    
*   cloud configuration
    
*   storage
    
*   AI endpoints
    
*   RAG
    
*   agents
    
*   media processing
    
*   tenant isolation
    

Testing should be performed in a controlled and authorized environment.

The objective is to identify weaknesses and improve defenses, not to disrupt production services.

* * *

# 70.30 Security Test Environment

A dedicated security-testing environment is recommended.

Example:

```text
Production
    │
    │
    └── isolated
         ↓
Security/Staging
         ↓
Synthetic data
         ↓
Test identities
         ↓
Test API keys
         ↓
Test storage
         ↓
Test models
```

Production credentials should not be reused unnecessarily.

* * *

# 70.31 Synthetic Security Data

Security testing should preferably use synthetic or controlled data.

Examples:

```text
TEST_USER_A
TEST_USER_B
TEST_TENANT_A
TEST_TENANT_B
TEST_DOCUMENT_A
TEST_DOCUMENT_B
```

This makes cross-user and cross-tenant testing easier without exposing real personal information.

* * *

# 70.32 Security Test Data Isolation

Security-test data should be isolated from production.

A safe design is:

```text
Production database
        X
        |
        | no direct test writes
        |
Security test database
```

Test automation should have its own:

*   credentials
    
*   storage
    
*   queues
    
*   databases
    
*   API keys
    
*   AI provider projects where possible
    

* * *

# 70.33 Performance and Security Testing

Security controls can introduce performance costs.

Examples include:

*   malware scanning
    
*   encryption
    
*   policy checks
    
*   authorization queries
    
*   rate limiting
    
*   logging
    
*   AI safety evaluation
    

Testing should measure:

```text
Security enabled
vs.
Security disabled
```

without sacrificing required security controls simply to improve performance.

Instead, optimize implementation.

* * *

# 70.34 Resource-Exhaustion Testing

AI workloads can be computationally expensive.

Testing should examine:

*   huge prompts
    
*   oversized uploads
    
*   long conversations
    
*   excessive retrieval requests
    
*   repeated generation
    
*   large media files
    
*   expensive agent loops
    

Controls may include:

```text
Request limits
Token limits
File-size limits
Timeouts
Concurrency limits
Quota limits
Budget limits
Maximum agent steps
```

The goal is to prevent a single request from consuming disproportionate resources.

* * *

# 70.35 Chaos and Failure Testing

Security includes resilience.

Controlled failure testing can verify:

*   database outage
    
*   storage outage
    
*   model-provider outage
    
*   queue failure
    
*   cache failure
    
*   authentication-service failure
    
*   key-management failure
    

The system should fail safely.

For example:

```text
Authorization service unavailable
        ↓
Protected action
        ↓
DENY
```

rather than:

```text
Authorization service unavailable
        ↓
Assume allowed
        ↓
Execute
```

For security-sensitive operations, fail-closed behavior is often preferable.

* * *

# 70.36 Cryptographic Verification Testing

Cryptographic functionality should be tested for:

*   correct algorithm selection
    
*   key handling
    
*   nonce uniqueness requirements
    
*   authentication-tag verification
    
*   signature verification
    
*   certificate validation
    
*   key rotation
    
*   expired-key handling
    
*   decryption failure behavior
    

Testing should use established cryptographic libraries rather than custom algorithms.

* * *

# 70.37 Audit Logging Tests

Security events should be tested for logging.

Examples:

```text
Login failure
Login success
Password reset
Permission denial
File access
File deletion
API-key creation
API-key revocation
Agent action
Tool denial
Administrative action
Policy violation
```

Tests should verify both:

**event generation**

and

**event integrity/privacy.**

Sensitive credentials should not appear in logs.

* * *

# 70.38 Detection Engineering Tests

A security control is incomplete if attacks can occur without detection.

Example:

```text
Repeated failed authentication
        ↓
Detection rule
        ↓
Alert
        ↓
Incident workflow
```

Testing should confirm that the alert actually fires.

This can be called a **detection test** or **security control validation**.

* * *

# 70.39 Incident Response Exercises

Security testing should extend beyond vulnerability discovery.

Organizations should periodically simulate scenarios such as:

```text
Compromised API key
Stolen session
Unauthorized data access
Malicious uploaded file
AI prompt injection
Cloud credential exposure
Provider outage
Database corruption
```

The objective is to verify:

*   detection
    
*   escalation
    
*   containment
    
*   credential rotation
    
*   communication
    
*   recovery
    
*   evidence preservation
    

* * *

# 70.40 Security Evidence

A mature system should retain evidence of security testing.

Examples:

```text
Test results
Scan reports
Dependency reports
SBOMs
Threat models
Penetration-test reports
AI red-team results
Regression-test results
Remediation records
Approval records
Deployment evidence
```

This creates an auditable security history.

* * *

# 70.41 Security Testing Metrics

Useful metrics include:

### Vulnerability metrics

*   critical findings
    
*   high findings
    
*   mean time to remediation
    
*   reopened findings
    
*   overdue findings
    

### Testing metrics

*   security-test coverage
    
*   API coverage
    
*   authorization-test coverage
    
*   AI evaluation coverage
    
*   red-team scenario coverage
    

### Operational metrics

*   detection time
    
*   response time
    
*   containment time
    
*   recovery time
    

### Regression metrics

*   security failures introduced per release
    
*   security failures caught before production
    
*   security failures discovered after deployment
    

Metrics should measure meaningful risk reduction rather than merely the number of scans executed.

* * *

# 70.42 Security Test Matrix

A useful testing matrix is:

| Component | Static | Dynamic | Adversarial | Regression |
| --- | --- | --- | --- | --- |
| Frontend | ✓ | ✓ | ✓ | ✓ |
| API | ✓ | ✓ | ✓ | ✓ |
| Database | ✓ | ✓ | ✓ | ✓ |
| Storage | ✓ | ✓ | ✓ | ✓ |
| RAG | ✓ | ✓ | ✓ | ✓ |
| AI model | Limited | ✓ | ✓ | ✓ |
| Agents | Limited | ✓ | ✓ | ✓ |
| Media pipeline | ✓ | ✓ | ✓ | ✓ |
| Infrastructure | ✓ | ✓ | ✓ | ✓ |
| Authentication | ✓ | ✓ | ✓ | ✓ |

* * *

# 70.43 Security Test Pipeline for an AI Platform

A practical pipeline can look like:

```text
Developer Commit
      ↓
Secret Scan
      ↓
SAST
      ↓
SCA
      ↓
Unit Tests
      ↓
Security Unit Tests
      ↓
IaC Scan
      ↓
Container Scan
      ↓
Build
      ↓
Integration Tests
      ↓
API Security Tests
      ↓
DAST
      ↓
RAG Security Tests
      ↓
Agent Security Tests
      ↓
AI Red-Team Regression Tests
      ↓
Performance / Resource Tests
      ↓
Security Approval
      ↓
Deployment
      ↓
Runtime Monitoring
```

* * *

# 70.44 AI Security Evaluation Dataset

An AI application should maintain a version-controlled security evaluation set.

Example:

```text
security-evals/
    authentication/
    authorization/
    prompt-injection/
    jailbreak/
    privacy/
    data-leakage/
    rag/
    agents/
    tools/
    multilingual/
    multimodal/
    output-safety/
```

Each test can contain:

```text
test_id
category
input
expected_behavior
risk_level
actual_behavior
pass/fail
model_version
application_version
timestamp
```

This makes AI security measurable over time.

* * *

# 70.45 Model Regression Testing

Changing the model can change security behavior.

For example:

```text
Model A
→ refuses attack scenario

Model B
→ provides unsafe output
```

The application code did not change, but the security posture did.

Therefore every model upgrade should trigger:

*   safety evaluation
    
*   prompt-injection testing
    
*   privacy testing
    
*   tool-use testing
    
*   multilingual testing
    
*   regression testing
    

Model deployment should therefore be treated as a security-relevant change.

* * *

# 70.46 Provider Regression Testing

Changing AI providers can also change behavior.

For example:

```text
Provider A
      ↓
specific refusal behavior

Provider B
      ↓
different output behavior
```

The application should not assume equivalent security properties across providers.

The provider abstraction layer should therefore have provider-specific evaluation suites.

* * *

# 70.47 Continuous Assurance Architecture

A mature security architecture can be represented as:

```text
             ┌─────────────────────┐
             │   Threat Modeling   │
             └──────────┬──────────┘
                        ↓
             ┌─────────────────────┐
             │ Security Controls   │
             └──────────┬──────────┘
                        ↓
       ┌────────────────────────────────┐
       │        Security Testing        │
       │                                │
       │ SAST / SCA / DAST / Fuzzing   │
       │ IaC / Container / API         │
       │ AI Red Team / RAG / Agents    │
       └────────────────┬───────────────┘
                        ↓
             ┌─────────────────────┐
             │ Findings & Evidence │
             └──────────┬──────────┘
                        ↓
             ┌─────────────────────┐
             │ Remediation         │
             └──────────┬──────────┘
                        ↓
             ┌─────────────────────┐
             │ Regression Testing  │
             └──────────┬──────────┘
                        ↓
             ┌─────────────────────┐
             │ Production Monitor  │
             └──────────┬──────────┘
                        │
                        └──────→ Feedback
```

This creates a continuous security loop.

* * *

# 70.48 Security Testing Principles

The most important principles are:

### Principle 1 — Test controls, not just code

A feature can work correctly while violating security requirements.

### Principle 2 — Test authorization explicitly

Authentication proves identity. Authorization determines access.

### Principle 3 — Treat AI output as untrusted

Generated content must pass appropriate validation.

### Principle 4 — Treat retrieved content as untrusted

Documents and webpages can contain malicious instructions.

### Principle 5 — Test agents as privileged systems

Tool access requires explicit permission boundaries.

### Principle 6 — Test every deployment

Security is not a one-time activity.

### Principle 7 — Test model changes

Changing a model can change security behavior.

### Principle 8 — Test failure modes

A system must fail safely.

### Principle 9 — Automate critical tests

Important security guarantees should not depend entirely on manual testing.

### Principle 10 — Preserve evidence

Security decisions should be reproducible and auditable.

* * *

# 70.49 Production Security Verification Checklist

Before production:

```text
[ ] SAST completed
[ ] SCA completed
[ ] Secret scan completed
[ ] IaC scan completed
[ ] Container scan completed
[ ] API security tests completed
[ ] DAST completed
[ ] Authentication tests completed
[ ] Authorization tests completed
[ ] Tenant isolation tested
[ ] File upload security tested
[ ] RAG security tested
[ ] Prompt injection tested
[ ] AI red-team tests completed
[ ] Agent permissions tested
[ ] Tool authorization tested
[ ] Multilingual security tested
[ ] Multimodal security tested
[ ] Resource limits tested
[ ] Logging tested
[ ] Detection tested
[ ] Incident response tested
[ ] Backup/recovery tested
[ ] Model regression tests completed
[ ] Provider regression tests completed
[ ] Critical findings resolved
[ ] Security approval recorded
```

* * *

# 70.50 Final Architecture

The complete security-assurance lifecycle is:

```text
Threat Modeling
      ↓
Security Requirements
      ↓
Secure Design
      ↓
Secure Implementation
      ↓
Automated Security Testing
      ↓
AI Red Teaming
      ↓
Penetration Testing
      ↓
Remediation
      ↓
Regression Testing
      ↓
Security Approval
      ↓
Production Deployment
      ↓
Runtime Detection
      ↓
Incident Response
      ↓
Post-Incident Learning
      ↓
Updated Threat Model
      ↺
```

Security testing is therefore not a final checklist performed immediately before launch.

It is a continuous engineering discipline.

For AI applications, the strongest approach combines conventional application security, infrastructure security, privacy testing, adversarial AI evaluation, agent testing, multimodal testing, and continuous runtime assurance.

The fundamental principle is:

**A security control should not be considered effective until its intended security property has been tested, the result has been measured, and the test can be repeated after future changes.**