Smoke Test

Exercise the application directly to verify behavior beyond automated tests. This command performs agent-driven manual and exploratory verification to confirm features work end-to-end in a real environment.

Additional instructions from the user: "$ARGUMENTS". Ignore if empty.

This command has four phases. Complete all four in order.

Phase 1: Discover

1. Identify the application type:
   • CLI tool: Look for binary entry points, bin fields in package files, or executable scripts.
   • Web API: Look for server entry points, route definitions, or API framework configuration.
   • Library: Look for exported modules, public API surface, or package entry points.
   • Web UI: Look for frontend frameworks, HTML entry points, or dev server configuration.
2. Determine how to build and run the application (build scripts, dev server commands, entry points).
3. Identify the key features and workflows to exercise based on E2E_TESTS.md specs, README, and source code.
4. Check for any required environment setup (environment variables, databases, dependencies).

Phase 2: Exercise

Run the application and interact with it based on its type:

CLI tools

• Run commands with typical inputs and verify expected stdout, stderr, and exit codes.
• Run with edge-case inputs (empty input, missing arguments, invalid flags).
• Run with --help and verify usage information is accurate.
• Test piping and redirection if applicable.

Libraries

• Write and execute small demo scripts in /tmp to test key exported functions and classes.
• Test with valid inputs, edge cases, and error-triggering inputs.
• Verify return types and error handling match documentation.
• Clean up demo scripts after execution.

Web APIs

• Start the dev server.
• Exercise endpoints with curl — test happy paths, error cases, and edge cases.
• Verify response status codes, headers, and body content.
• Test authentication and authorization flows if applicable.
• Stop the dev server when done.

Web UIs

• Start the dev server.
• Use Playwright or equivalent browser automation to load pages.
• Verify key UI elements are present and interactive.
• Test user workflows (navigation, form submission, state changes).
• Capture screenshots of key states if the tooling supports it.
• Stop the dev server when done.

Phase 3: Capture

Document what was tested and observed:

1. For each interaction, record:
   • The exact command or action performed
   • The input provided
   • The output or response received
   • Whether the behavior matched expectations
2. Note any screenshots captured (for UI testing).
3. Flag any unexpected behavior, errors, or discrepancies.

Phase 4: Report

Present findings to the user:

1. Summary: What type of application was tested and how many interactions were performed.
2. What works: Features that behave as expected.
3. What's broken: Features that fail, produce wrong output, or behave unexpectedly.
4. Gaps in automated tests: Behaviors observed during manual testing that are not covered by existing E2E_TESTS.md specs.
5. Suggested spec entries: For any discovered issues or uncovered behaviors, propose new entries for E2E_TESTS.md so they can be captured as automated tests.