Audit E2E Test Coverage
Analyze the gap between E2E specifications and existing test implementations. This is a read-only command — it reports findings but does not modify any files.
Additional instructions from the user: "$ARGUMENTS". Ignore if empty.
Phase 1: Discover
- Find all
E2E_TESTS.mdfiles by searching recursively from the project root. - Read the root-level spec first (
docs/E2E_TESTS.mdorE2E_TESTS.md) to understand project-wide testing constraints. - Read each package-level
E2E_TESTS.mdand extract:- All suites (H2 headings)
- All features within each suite (H4 headings)
- Category metadata (
<!-- category: ... -->) - Skip metadata (
<!-- skip: ... -->)
- Find all existing E2E test files:
- Files matching
**/*.e2e.test.* - Files inside
**/e2e/**/*.test.*
- Files matching
- Read existing test files and extract:
- Suite/group blocks — the framework's grouping construct (e.g.,
describe()in vitest/jest,mod testsin Rust,classin pytest) — map to suites - Individual test cases — the framework's test declaration (e.g.,
it()/test()in vitest/jest,#[test]in Rust,def test_in pytest) — map to features - Skip/conditional-skip markers — the framework's mechanism for skipping tests (e.g.,
skip/skipIfin vitest/jest,#[ignore]in Rust,@pytest.mark.skipin pytest)
- Suite/group blocks — the framework's grouping construct (e.g.,
Phase 2: Map
Build a coverage matrix:
| Spec Suite | Spec Feature | Category | Test File | Test Case | Status |
|---|---|---|---|---|---|
| Task Creation | Create basic task | core | task.e2e.test.ts | "Create basic task" | Covered |
| Task Creation | Create with long name | edge | — | — | Missing |
| Push | Push to remote | core | push.e2e.test.ts | "push to remote" | Covered |
For each spec feature, determine:
- Covered: A corresponding test case exists and its assertions match the spec
- Missing: No corresponding test case
- Stale: Test exists but assertions don't match current spec
- Skipped: Test exists with a skip marker (expected for
<!-- skip: ... -->features)
Phase 3: Report
Present findings in this format:
E2E Test Coverage Audit
========================
Specs found: N files
Test files found: N files
Overall coverage: X/Y features (Z%)
By category:
core: X/Y (Z%)
edge: X/Y (Z%)
error: X/Y (Z%)
side-effect: X/Y (Z%)
idempotency: X/Y (Z%)
Gaps by suite:
## Task Creation (3/5 features covered)
MISSING [edge] Create with very long name
MISSING [idempotency] Re-create after deletion
## Push (4/4 features covered)
(fully covered)
Stale tests:
push.e2e.test.ts: "push with pending changes"
Spec says: Exits with non-zero code
Test asserts: Exits with code 0
Structural issues:
- task.e2e.test.ts: Tests share mutable state at suite level
- init.e2e.test.ts: Missing afterEach cleanup for env.HOME
Do NOT suggest fixes or modifications. This command is purely diagnostic.