Silk Debug Tool

A CLI tool for analyzing Django Silk profiling data to debug slow requests, detect N+1 queries, and optimize database performance.

Tool Location

.venv/bin/python scripts/silk_debug.py

Quick Reference

Analyze a Specific Request

When given a Silk request ID (UUID), use --full for comprehensive analysis:

.venv/bin/python scripts/silk_debug.py <request_id> --full

This shows:

• Request info (path, method, status, time, query count)

• Duplicate/similar queries (N+1 detection)

• Slow queries (>5ms by default)

• Queries grouped by table

• Query execution timeline

• Python cProfile data (if enabled)

List and Filter Requests

List recent requests

.venv/bin/python scripts/silk_debug.py --list

Sort by different criteria

.venv/bin/python scripts/silk_debug.py --list --sort queries # Most queries .venv/bin/python scripts/silk_debug.py --list --sort duration # Slowest total time .venv/bin/python scripts/silk_debug.py --list --sort db_time # Most DB time

Filter requests

.venv/bin/python scripts/silk_debug.py --list --path /api/events --min-queries 20 .venv/bin/python scripts/silk_debug.py --list --method POST --min-time 100

Aggregate Analysis

Overall statistics

.venv/bin/python scripts/silk_debug.py --stats

Endpoint summary (grouped by path pattern, shows P95)

.venv/bin/python scripts/silk_debug.py --endpoints

Find slow endpoints

.venv/bin/python scripts/silk_debug.py --slow-endpoints --slow-endpoint-threshold 100

Interpreting Results

N+1 Query Detection

When you see duplicate queries like:

🔴 15x similar queries: SELECT "events_ticket"."id" FROM "events_ticket" WHERE "events_ticket"."event_id" = '<UUID>'

This indicates an N+1 problem. Fix with:

• select_related() for ForeignKey fields

• prefetch_related() for reverse relations or M2M fields

Slow Queries

Common causes of slow queries:

• COUNT on complex DISTINCT: Pagination wrapping complex visibility subqueries

• Fix: Materialize IDs in Python first, then filter with simple IN clause

• Missing indexes: Full table scans

• Fix: Add database indexes on filtered/joined columns

• Complex JOINs: Multiple related tables

• Fix: Optimize query structure or denormalize if appropriate

Timeline Analysis

The timeline shows query execution order with visual bars:

1. • 0.0ms [ 2.5ms] █ "accounts_reveluser"
2. • 60.2ms [ 2.8ms] █ "__count"

Look for:

• Large gaps between queries (indicates Python processing time)

• Queries that could run in parallel but are sequential

• Expensive queries that block subsequent operations

Common Optimization Patterns

Expensive COUNT with DISTINCT

When you see:

SELECT COUNT(*) FROM (SELECT DISTINCT ... complex subquery ...)

Fix by materializing IDs:

Before (slow COUNT)

qs = Event.objects.for_user(user).filter(...).distinct()

After (fast COUNT)

event_ids = list(Event.objects.for_user(user).values_list("id", flat=True).distinct()) qs = Event.objects.full().filter(id__in=event_ids)

Redundant Visibility Checks

When the same for_user() query appears multiple times:

• Create a method that accepts already-checked objects

• Cache visibility results within the request

Batch Operations

When creating multiple objects:

• Use bulk_create() instead of individual .save() calls

• Fetch shared data (settings, related objects) once before the loop

• Send notifications in batches, not per-item

CLI Options Reference

Single Request Analysis

• --full, -f : Run all analyses

• --duplicates, -d : Show N+1 candidates

• --slow, -s : Show slow queries

• --slow-threshold N : Slow query threshold in ms (default: 5)

• --tables, -t : Group queries by table

• --timeline : Show execution timeline

• --traceback, -tb : Show code locations for duplicates

• --profile, -prof : Show Python cProfile data

Listing and Filtering

• --list, -l : List requests

• --limit N : Number of results (default: 20)

• --sort {recent,queries,duration,db_time} : Sort order

• --path, -p : Filter by path (contains)

• --method, -m : Filter by HTTP method

• --status : Filter by status code

• --min-queries N : Minimum query count

• --min-time N : Minimum response time (ms)

• --min-db-time N : Minimum DB time (ms)

Aggregate Views

• --stats : Show aggregate statistics

• --endpoints : Show endpoint summary with P95

• --slow-endpoints : Group slow requests by endpoint

• --slow-endpoint-threshold N : Threshold in ms (default: 200)

• --min-count N : Minimum requests for endpoint summary