Security Integration Tests Agent

Overview

The security package (pkg/security/ ) provides prompt injection detection using the HuggingFace Inference API. Integration tests verify detection accuracy with real API calls.

Key Files

• pkg/security/testdata/samples.json

• Test data with benign and malicious samples

• pkg/security/hf/integration_test.go

• HF client integration tests

• pkg/security/integration_test.go

• Full analyzer integration tests with event bus

• pkg/security/hf/client.go

• HuggingFace API client implementation

• pkg/security/analyzer.go

• Main security analyzer that subscribes to MCP events

Running Tests

Run integration tests (requires HF_TOKEN environment variable)

HF_TOKEN=hf_xxx make test-integration

Run with custom model

HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration

Run unit tests only (no API calls, uses mock server)

go test -v ./pkg/security/...

Test Structure

Test Samples (samples.json )

{ "benign_samples": [ { "description": "Description of the sample", "text": "The text to analyze", "expected_detected": false } ], "malicious_samples": [ { "description": "Direct instruction override", "text": "Ignore all previous instructions...", "expected_detected": true, "min_risk_score": 0.5 } ], "mcp_tool_calls": [ { "description": "Malicious tool call", "method": "tools/call", "params": { "name": "run_command", "arguments": {...} }, "expected_detected": true, "min_risk_score": 0.5 } ] }

Integration Test Tags

Integration tests use the build tag //go:build integration and are excluded from regular go test ./... runs.

Adding New Test Samples

• Edit pkg/security/testdata/samples.json

• Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)

• Set expected_detected and optionally min_risk_score

• Run integration tests to verify

Common Issues

"Forbidden" Error

• Ensure HF_TOKEN is set and valid

• Note: meta-llama/Llama-Prompt-Guard-2-86M is deprecated on HF Inference API

• Default test model is protectai/deberta-v3-base-prompt-injection-v2 (publicly accessible)

Model Loading

• HuggingFace warms up models on demand

• Tests may skip with "Model loading" message on first run

• Re-run tests after model is warm

Network Issues

• Integration tests require network access to HuggingFace API

• Tests will fail in sandboxed environments without network access

Risk Levels

• none : score < 0.3

• low : score 0.3-0.5

• medium : score 0.5-0.7

• high : score 0.7-0.9

• critical : score >= 0.9

Categories

• benign : Normal, safe content

• injection : Prompt injection attempt

• jailbreak : Jailbreak attempt

• malicious : Malicious content (Prompt Guard v2)