Sidekiq Job Analyzer
Analyze Sidekiq job configurations for reliability, performance, queue design, retry policies, and memory safety. Audit worker classes, queue priorities, concurrency settings, and Redis usage. Use when jobs are slow, failing silently, or consuming too much memory.
Usage
"Analyze my Sidekiq jobs for issues"
"Check queue configuration and priorities"
"Audit retry policies and error handling"
"Optimize Sidekiq performance and memory usage"
How It Works
1. Job Discovery
# Find all Sidekiq workers/jobs
grep -rn "include Sidekiq::Job\|include Sidekiq::Worker" app/ lib/ | head -30
# Check Sidekiq config
cat config/sidekiq.yml 2>/dev/null
cat config/initializers/sidekiq.rb 2>/dev/null
# Check queue definitions
grep -rn "sidekiq_options" app/ lib/ | head -20
2. Job Configuration Audit
Retry policies:
- Jobs without explicit retry count (default: 25 retries over 21 days)
- Missing
dead: falseon non-critical jobs - No custom error handling or
sidekiq_retries_exhaustedcallback - Retry with exponential backoff vs fixed intervals
Queue design:
- Too many queues (scheduler overhead)
- Critical jobs sharing queues with bulk operations
- Missing queue weights/priorities
- Jobs in wrong queues (heavy job in
defaultblocking lightweight jobs)
Idempotency:
- Jobs that aren't safe to retry (double-sends, duplicate charges)
- Missing unique job constraints (sidekiq-unique-jobs)
- State mutations without proper locking
Performance:
- Long-running jobs blocking the queue
- Jobs that should be batched (Sidekiq Pro/Enterprise)
- Missing
poolsize for database-heavy workers - Thread-safety issues in job code
3. Memory Analysis
- Jobs loading large datasets into memory
- Missing
find_each/find_in_batchesfor ActiveRecord queries - String concatenation in loops (memory fragmentation)
- Jobs that grow memory without bound
- Redis memory usage by queues and retry sets
4. Monitoring & Observability
- Sidekiq Web UI configured and accessible?
- Dead job monitoring and alerting
- Queue latency monitoring
- Error tracking integration (Sentry, Bugsnag)
- Custom metrics for business-critical jobs
5. Redis Configuration
- Redis connection pool sizing
- Redis memory limits and eviction policy
- Redis persistence settings for job durability
- Sentinel/Cluster for high availability
Output
## Sidekiq Job Analysis
**Workers:** 23 | **Queues:** 5 | **Redis:** 512MB used
### 🔴 Critical (2)
1. **Non-idempotent payment job** — PaymentProcessor has no uniqueness guard
If retried, can charge customer multiple times
→ Add `sidekiq_options unique: :until_executed`
2. **Memory leak in ReportGenerator** — loads all records per tenant
Grows to 2GB+ for large tenants, triggers OOM kill
→ Use `find_in_batches(batch_size: 1000)` and stream output
### 🟡 Improvements (4)
3. Default retry count (25) on BulkEmailJob — 21 days of retries for marketing emails
4. Queue `default` has 15 different job types — split critical from bulk
5. No `sidekiq_retries_exhausted` handler on 18/23 workers
6. Missing dead job monitoring (12,847 dead jobs accumulated)
### 📊 Queue Health
| Queue | Jobs/hr | Latency | Workers |
|-------|---------|---------|---------|
| critical | 120 | 0.1s | 5 |
| default | 3,400 | 45s ⚠️ | 10 |
| mailers | 800 | 2s | 3 |
| reports | 50 | 5min ⚠️ | 2 |
| bulk | 12,000 | 30min | 5 |
### ✅ Good Practices
- Separate queues for different priorities
- Sentry integration for error tracking
- Redis Sentinel configured for HA