Mutation Testing

Expert knowledge for mutation testing - validating that your tests actually catch bugs by introducing deliberate code mutations.

When to Use This Skill

Use this skill when... Use another skill instead when...

Validating test effectiveness Writing unit tests (use vitest-testing)

Finding weak/insufficient tests Analyzing test smells (use test-quality-analysis)

Setting up Stryker or mutmut Writing E2E tests (use playwright-testing)

Improving mutation score Generating test data (use property-based-testing)

Checking if tests catch real bugs Setting up code coverage only

Core Expertise

Mutation Testing Concept

• Mutants: Small code changes (mutations) introduced automatically

• Killed: Test fails with mutation (good - test caught the bug)

• Survived: Test passes with mutation (bad - weak test)

• Coverage: Tests execute mutated code but don't catch it

• Score: Percentage of mutants killed (aim for 80%+)

What Mutation Testing Reveals

• Tests that don't actually verify behavior

• Missing assertions or edge cases

• Overly permissive assertions

• Dead code or unnecessary logic

• Areas needing stronger tests

TypeScript/JavaScript (Stryker)

Installation

Using Bun

bun add -d @stryker-mutator/core

For Vitest

bun add -d @stryker-mutator/vitest-runner

For Jest

bun add -d @stryker-mutator/jest-runner

Running Stryker

npx stryker run # Run mutation testing npx stryker run --incremental # Only changed files npx stryker run --mutate "src/utils/**/*.ts" # Specific files npx stryker run --reporters html,clear-text # HTML report open reports/mutation/html/index.html # View report

Understanding Results

Mutation score: 82.5%

• Killed: 66 (tests caught the mutation)
• Survived: 14 (tests passed despite mutation - weak tests!)
• No Coverage: 0 (mutated code not executed)
• Timeout: 0 (tests took too long)

Example: Weak vs Strong Test

// Source code function calculateDiscount(price: number, percentage: number): number { return price - (price * percentage / 100) }

// WEAK: Test passes even if we mutate the calculation test('applies discount', () => { const result = calculateDiscount(100, 10) expect(result).toBeDefined() // Too weak! })

// STRONG: Test catches mutation test('applies discount correctly', () => { expect(calculateDiscount(100, 10)).toBe(90) expect(calculateDiscount(100, 20)).toBe(80) expect(calculateDiscount(50, 10)).toBe(45) })

Python (mutmut)

Installation

uv add --dev mutmut # Using uv pip install mutmut # Using pip

Running mutmut

uv run mutmut run # Run mutation testing uv run mutmut run --paths-to-mutate=src/calculator.py # Specific files uv run mutmut results # Show results uv run mutmut summary # Summary uv run mutmut show 1 # Show specific mutant uv run mutmut apply 1 # Apply mutant manually uv run mutmut html # HTML report

Understanding Results

Status: 45/50 mutants killed (90%)

• Killed: 45 (tests caught the mutation)
• Survived: 5 (tests passed despite mutation)

Mutation Score Targets

Score Quality Action

90%+ Excellent Maintain quality

80-89% Good Small improvements

70-79% Acceptable Focus on weak areas

60-69% Needs work Add missing tests

< 60% Poor Major test improvements needed

Agentic Optimizations

Context Command

Quick TS mutation npx stryker run --incremental --reporters clear-text

Targeted TS mutation npx stryker run --mutate "src/core/**/*.ts"

Quick Python mutation uv run mutmut run --paths-to-mutate=src/core/

View survived uv run mutmut results | grep Survived

CI mode npx stryker run --reporters json

For detailed examples, advanced patterns, and best practices, see REFERENCE.md.

See Also

• vitest-testing

• Unit testing framework

• python-testing

• Python pytest testing

• test-quality-analysis

• Detecting test smells

• api-testing

• HTTP API testing

References

• Stryker: https://stryker-mutator.io/

• mutmut: https://github.com/boxed/mutmut

• Mutation Testing Intro: https://en.wikipedia.org/wiki/Mutation_testing