QA Testing Strategy (Jan 2026)

Risk-based quality engineering strategy for modern software delivery.

Core references: curated links in data/sources.json (SLOs/error budgets, contracts, E2E, OpenTelemetry). Start with references/operational-playbook.md for a compact, navigable overview.

Scope

• Create or update a risk-based test strategy (what to test, where, and why)

• Define quality gates and release criteria (merge vs deploy)

• Select the smallest effective layer (unit → integration → contract → E2E)

• Make failures diagnosable (artifacts, logs/traces, ownership)

• Operationalize reliability (flake SLO, quarantines, suite budgets)

Use Instead

Need Skill

Debug failing tests or incidents qa-debugging

Test LLM agents/personas qa-agent-testing

Perform security audit/threat model software-security-appsec

Design CI/CD pipelines and infra ops-devops-platform

Quick Reference

Test Type Goal Typical Use

Unit Prove logic and invariants fast Pure functions, core business rules

Component Validate UI behavior in isolation UI components and state transitions

Integration Validate boundaries with real deps API + DB, queues, external adapters

Contract Prevent breaking changes cross-team OpenAPI/AsyncAPI/JSON Schema/Protobuf

E2E Validate critical user journeys 1–2 “money paths” per product area

Performance Enforce budgets and capacity Load, stress, soak, regression trends

Visual Catch UI regressions Layout/visual diffs on stable pages

Accessibility Automate WCAG checks axe smoke + targeted manual audits

Security Catch common web vulns early DAST smoke + critical checks in CI

Default Workflow

• Clarify scope and risk: critical journeys, failure modes, and non-functional risks (latency, data loss, auth).

• Define quality signals: SLOs/error budgets, contract/schema checks, and what blocks merge vs blocks deploy.

• Choose the smallest effective layer (unit → integration → contract → E2E).

• Make failures diagnosable: artifacts + correlation IDs (logs/traces/screenshots), clear ownership, deflake runbook.

• Operationalize: flake SLO, quarantine with expiry, suite budgets (PR gate vs scheduled), dashboards.

Test Pyramid

       /\
      /E2E\          5-10% - Critical journeys
     /------\
    /Integr. \       15-25% - API, DB, queues
   /----------\
  /Component \       20-30% - UI modules
 /------------\
/   Unit      \      40-60% - Logic and invariants

/--------------\

Decision Tree: Test Strategy

Need to test: [Feature Type] │ ├─ Pure business logic/invariants? → Unit tests (mock boundaries) │ ├─ UI component/state transitions? → Component tests │ └─ Cross-page user journey? → E2E tests │ ├─ API Endpoint? │ ├─ Single service boundary? → Integration tests (real DB/deps) │ └─ Cross-service compatibility? → Contract tests (schema/versioning) │ ├─ Event-driven/API schema evolution? → Contract + backward-compat tests │ └─ Performance-critical? → k6 load testing

Core QA Principles

Definition of Done

• Strategy is risk-based: critical journeys + failure modes explicit

• Test portfolio is layered: fast checks catch most defects

• CI is economical: fast pre-merge gates, heavy suites scheduled

• Failures are diagnosable: actionable artifacts (logs/trace/screenshots)

• Flakes managed with SLO and deflake runbook

Shift-Left Gates (Pre-Merge)

• Contracts: OpenAPI/AsyncAPI/JSON Schema validation

• Static checks: lint, typecheck, secret scanning

• Fast tests: unit + key integration (avoid full E2E as PR gate)

Shift-Right (Post-Deploy)

• Synthetic checks for critical paths (monitoring-as-tests)

• Canary analysis: compare SLO signals and key metrics before ramping

• Feature flags for safe rollouts and fast rollback

• Convert incidents into regression tests (prefer lower layers first)

CI Economics

Budget Target

PR gate p50 ≤ 10 min, p95 ≤ 20 min

Mainline health ≥ 99% green builds/day

Flake Management

• Define: test fails without product change, passes on rerun

• Track weekly: flaky_failures / total_test_executions (where flaky_failure = fail_then_pass_on_rerun )

• SLO: Suite flake rate ≤ 1% weekly

• Quarantine policy with owner and expiry

• Use the deflake runbook: template-flaky-test-triage-deflake-runbook.md

Common Patterns

AAA Pattern

it('should apply discount', () => { // Arrange const order = { total: 150 }; // Act const result = calculateDiscount(order); // Assert expect(result.discount).toBe(15); });

Page Object Model (E2E)

class LoginPage { async login(email: string, password: string) { await this.page.fill('[data-testid="email"]', email); await this.page.fill('[data-testid="password"]', password); await this.page.click('[data-testid="submit"]'); } }

Anti-Patterns

Anti-Pattern Problem Solution

Testing implementation Breaks on refactor Test behavior

Shared mutable state Flaky tests Isolate test data

sleep() in tests Slow, unreliable Use proper waits

Everything E2E Slow, expensive Use test pyramid

Ignoring flaky tests False confidence Fix or quarantine

Do / Avoid

Do

• Write tests against stable contracts and user-visible behavior

• Treat flaky tests as P1 reliability work

• Make "how to debug this failure" part of every suite

Avoid

• "Everything E2E" as default

• Sleeps/time-based waits (use event-based)

• Coverage % as primary quality KPI

Feature Matrix vs Test Matrix Gate (Release Blocking)

Before release, run a coverage audit that maps product features/backlog IDs to direct test evidence.

Gate Rules

• Every release-scoped feature must map to at least one direct automated test, or an explicit waiver with owner/date.

• Evidence must include file path and test identifier (suite/spec/case).

• "Covered indirectly" is not accepted without written rationale and risk acknowledgment.

• If critical features have no direct evidence, release is blocked.

Minimal Audit Output

• feature/backlog id

• coverage status (direct , indirect , none )

• evidence reference

• risk level

• owner and due date for gaps

Resources

Resource Purpose

comprehensive-testing-guide.md End-to-end playbook across layers

operational-playbook.md Testing pyramid, BDD, CI gates

shift-left-testing.md Contract-first, BDD, continuous testing

test-automation-patterns.md Reliable patterns and anti-patterns

playwright-webapp-testing.md Playwright patterns

chaos-resilience-testing.md Chaos engineering

observability-driven-testing.md OpenTelemetry, trace-based

contract-testing-2026.md Pact, Specmatic

synthetic-test-data.md Privacy-safe, ephemeral test data

test-environment-management.md Environment provisioning and lifecycle

quality-metrics-dashboard.md Quality metrics and dashboards

compliance-testing.md SOC2, HIPAA, GDPR, PCI-DSS testing

feature-matrix-vs-test-matrix-gate.md Release-blocking feature-to-test coverage audit

Templates

Template Purpose

template-test-case-design.md Given/When/Then and test oracles

test-strategy-template.md Risk-based strategy

template-flaky-test-triage.md Flake triage runbook

template-jest-vitest.md Unit test patterns

template-api-integration.md API + DB integration tests

template-playwright.md Playwright E2E

template-visual-testing.md Visual regression testing

template-k6-load-testing.md k6 performance

automation-pipeline-template.md CI stages, budgets, gates

template-cucumber-gherkin.md BDD feature files and steps

template-release-coverage-audit.md Feature matrix vs test matrix release audit

Data

File Purpose

sources.json External references

Related Skills

• qa-debugging — Debugging failing tests

• qa-agent-testing — Testing AI agents

• software-backend — API patterns to test

• ops-devops-platform — CI/CD pipelines

Ops Gate: Release-Safe Verification Sequence

Use this sequence for feature branches that touch user flows, pricing, localization, or analytics.

1) Static checks

npm run lint npm run typecheck

2) Fast correctness

npm run test:unit

3) Critical path checks

npm run test:e2e -- --grep "@critical"

4) Instrumentation gate (if configured)

npm run test:analytics-gate

5) Production build

npm run build

If a Gate Fails

• Capture exact failing command and first error line.

• Classify: environment issue, baseline known failure, or regression.

• Re-run only the failed gate once after fix.

• Do not continue to later gates while earlier required gates are red.

Agent Output Contract for QA Handoff

Always report:

• commands run,

• pass/fail per gate,

• whether failures are pre-existing or introduced,

• next blocking action.

Fact-Checking

• Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.

• Prefer primary sources; report source links and dates for volatile information.

• If web access is unavailable, state the limitation and mark guidance as unverified.