Report Comparison

This skill compares two security audit reports to track progress over time.

When to Use This Skill

• After fixing vulnerabilities, to verify remediation

• For periodic security reviews

• To track security posture over time

• To identify regression (new vulnerabilities)

Prerequisites

• Two audit reports in Markdown format

• Reports should be from the same project

Usage

Basic Comparison

Compare security reports old-report.md and new-report.md

With Specific Paths

Compare reports/audit-v1.md with reports/audit-v2.md

Output Format

═══════════════════════════════════════════════════════════ SECURITY AUDIT COMPARISON ═══════════════════════════════════════════════════════════

Previous Audit: January 15, 2025 Current Audit: January 31, 2025 Days Between: 16 days

───────────────────────────────────────────────────────── Score Comparison ─────────────────────────────────────────────────────────

Previous Score: 35/100 (Grade: D) Current Score: 72/100 (Grade: C) Improvement: +37 points ⬆️

┌────────────────────────────────────────────────────────┐ │ Score Progress │ │ │ │ 100 ┤ │ │ 80 ┤ ████████ 72 │ │ 60 ┤ ████████ │ │ 40 ┤ ████████ 35 ████████ │ │ 20 ┤ ████████ ████████ │ │ 0 ┴─────────────────────────────────────────────── │ │ Jan 15 Jan 31 │ └────────────────────────────────────────────────────────┘

───────────────────────────────────────────────────────── Findings Summary ─────────────────────────────────────────────────────────

───────────────────────────────────────────────────────── Fixed Vulnerabilities ✅ ─────────────────────────────────────────────────────────

P0 (Critical) - ALL FIXED! 🎉

✅ P0-001: Service Role Key Exposed Status: FIXED Resolution: Key rotated, removed from client code Fixed on: January 16, 2025

✅ P0-002: Database Backups Publicly Accessible Status: FIXED Resolution: Bucket made private, files deleted Fixed on: January 16, 2025

✅ P0-003: Admin Function Privilege Escalation Status: FIXED Resolution: Added admin role verification Fixed on: January 17, 2025

P1 (High) - 2 of 4 Fixed

✅ P1-001: Email Confirmation Disabled Status: FIXED Resolution: Email confirmation now required Fixed on: January 20, 2025

✅ P1-002: IDOR in get-user-data Function Status: FIXED Resolution: Added user ownership verification Fixed on: January 18, 2025

P2 (Medium) - 2 of 5 Fixed

✅ P2-001: Weak Password Policy Status: FIXED Resolution: Minimum length increased to 10 Fixed on: January 22, 2025

✅ P2-003: Disposable Emails Accepted Status: FIXED Resolution: Email validation added Fixed on: January 25, 2025

───────────────────────────────────────────────────────── Remaining Vulnerabilities ⚠️ ─────────────────────────────────────────────────────────

P1 (High) - 2 Remaining

🟠 P1-003: User Enumeration via Timing Attack Status: OPEN (16 days) Priority: Address this week Note: Was in previous report, not yet fixed

🟠 P1-004: Admin Channel Publicly Accessible Status: OPEN (16 days) Priority: Address this week

P2 (Medium) - 3 Remaining

🟡 P2-002: Wildcard CORS Origin Status: OPEN (16 days)

🟡 P2-004: Verbose Error Messages Status: OPEN (16 days)

🟡 P2-005: Rate Limiting Not Enforced on Functions Status: OPEN (16 days)

───────────────────────────────────────────────────────── New Vulnerabilities 🆕 ─────────────────────────────────────────────────────────

P2 (Medium) - 1 New Issue

🆕 P2-006: New Storage Bucket Without RLS Severity: 🟡 P2 Component: Storage Description: New bucket 'user-uploads' created without RLS policies. Currently empty but will need policies before production use. First Seen: January 31, 2025

───────────────────────────────────────────────────────── Progress Analysis ─────────────────────────────────────────────────────────

Remediation Rate: 58% (7 of 12 fixed)

By Severity: ├── P0 (Critical): 100% fixed ✅ ├── P1 (High): 50% fixed └── P2 (Medium): 40% fixed

Time to Fix (Average): ├── P0: 1.3 days (excellent) ├── P1: 3.5 days (good) └── P2: 5.5 days (acceptable)

Regression: 1 new issue introduced (lower severity, acceptable)

───────────────────────────────────────────────────────── Recommendations ─────────────────────────────────────────────────────────

1. CONTINUE PROGRESS Great work fixing all P0 issues! Focus now on remaining P1 issues:

   • User enumeration timing attack
   • Admin broadcast channel

2. ADDRESS NEW ISSUE Configure RLS on 'user-uploads' bucket before it's used in production.

3. SCHEDULE FOLLOW-UP Recommend another audit in 14 days to verify remaining fixes.

───────────────────────────────────────────────────────── Trend Analysis ─────────────────────────────────────────────────────────

If you have 3+ reports, trend analysis is available:

Trend: Improving ⬆️

═══════════════════════════════════════════════════════════

Comparison Logic

Finding Matching

Findings are matched between reports using:

• ID match — Same P0-001, P1-002, etc.

• Component + Title match — Same issue description

• Location match — Same file/line/endpoint

Status Determination

Previous Current Status

Present Absent Fixed ✅

Present Present Remaining ⚠️

Absent Present New 🆕

Absent Absent N/A

Score Calculation

Change = Current Score - Previous Score

Positive change = Improvement ⬆️ Negative change = Regression ⬇️ No change = Stable ➡️

Context Output

{ "comparison": { "previous_date": "2025-01-15", "current_date": "2025-01-31", "previous_score": 35, "current_score": 72, "score_change": 37, "findings": { "previous_total": 12, "current_total": 6, "fixed": 7, "remaining": 5, "new": 1 }, "by_severity": { "P0": { "previous": 3, "current": 0, "fixed": 3, "new": 0 }, "P1": { "previous": 4, "current": 2, "fixed": 2, "new": 0 }, "P2": { "previous": 5, "current": 4, "fixed": 2, "new": 1 } }, "remediation_rate": 0.58, "trend": "improving" } }

Report Output

The comparison generates supabase-audit-comparison.md :

Security Audit Comparison Report

Summary

Fixed Issues (7)

[Detailed list of fixed issues...]

Remaining Issues (5)

[Detailed list of remaining issues...]

New Issues (1)

[Detailed list of new issues...]

Recommendations

[Action items based on comparison...]

Multiple Report Comparison

For trend analysis across 3+ reports:

Compare trend across reports/audit-*.md

Output includes:

• Score trend graph

• Issue count over time

• Average time to fix

• Recurring issues identification

Best Practices

Naming Convention

reports/ ├── supabase-audit-2024-12-01.md ├── supabase-audit-2025-01-15.md ├── supabase-audit-2025-01-31.md └── supabase-audit-comparison-2025-01-31.md

Regular Audits

Frequency Purpose

After fixes Verify remediation

Monthly Catch regressions

Before releases Pre-production check

After incidents Post-incident review

Tracking Progress

• Keep all reports in version control

• Link to issue tracker (GitHub, Jira)

• Include in sprint planning

• Report to stakeholders

Related Skills

• supabase-report — Generate the reports to compare

• supabase-pentest — Run full audit

• supabase-help — Quick reference