Update E2E Spec
Update an existing E2E_TESTS.md specification to reflect new features, changed behavior, or removed functionality. Uses git history and source code analysis to identify what changed.
Additional instructions from the user: "$ARGUMENTS". Ignore if empty.
Phase 1: Discover Changes
-
Find the target
E2E_TESTS.md:- If
$ARGUMENTSspecifies a path, use that. - If in a package directory with an
E2E_TESTS.md, use that. - Otherwise, ask the user.
- If
-
Read the current spec fully.
-
Analyze what changed since the spec was last updated:
From git history:
- Run
git logon the package's source files to find recent commits - Identify added, modified, and removed files
- Look for new commands, changed behavior, removed features
From source code:
- Compare the spec's suites and features against the current source code
- Look for new public APIs, commands, or entry points not covered by the spec
- Look for changed function signatures, options, or flags
- Look for removed or deprecated functionality still in the spec
- Run
Phase 2: Draft Updates
Categorize needed changes:
New features to add
For each new feature found:
- Determine which existing suite it belongs to (or if it needs a new suite)
- Draft the feature entry with category, assertions, and any skip metadata
- Draft updated preconditions if the new feature requires additional setup
Changed features to update
For each changed feature:
- Identify the current spec entry
- Draft the updated assertions to match the new behavior
- Note what changed and why
Removed features to flag
For each removed feature:
- Identify the spec entry
- Mark for removal
- Note: only remove if the functionality is truly gone, not just refactored
New suites to create
If entirely new commands or workflows were added:
- Draft complete suites using the canonical heading structure, with heading levels appropriate to their nesting depth: suite heading, then
Preconditions(one level below suite, with bullet list),Features(one level below suite) containing individual features as flat peer subheadings (one level below Features) — do NOT create intermediate category headings (e.g.,#### Core,#### Edge); categories are specified only via<!-- category: ... -->HTML comments on each feature — andPostconditions(one level below suite, with bullet list)
Phase 3: Review
Present the proposed changes to the user:
Spec Update Summary: packages/cli/E2E_TESTS.md
================================================
New features (5):
Suite "Task Creation":
+ [core] Create task with container image override
+ [error] Create task with invalid container image
NEW Suite "Configuration" (3 features):
+ [core] Read configuration from file
+ [error] Handle missing configuration file
+ [edge] Handle malformed configuration
Changed features (2):
Suite "Init":
~ [core] Initialize project — updated: now detects 3 additional languages
Removed features (1):
Suite "Push":
- [core] Push to SVN remote — SVN support was removed in commit abc123
No changes to:
Suite "Task Listing" (4 features)
Suite "Task Inspection" (6 features)
Ask the user to approve, adjust, or reject changes.
Phase 4: Apply
- Apply the approved changes to
E2E_TESTS.md. - Maintain the existing document structure and formatting.
- For new suites, place them in a logical position (typically at the end, or grouped with related suites) and add a link in the Index section. For removed suites, remove their link from the Index.
- Run
review-specvalidation on the updated file to ensure format correctness. - Inform the user they can now run:
updateto sync test files with the updated specauditto check the current test coverage against the new spec