Log Analysis

When to use this skill

• Error debugging: analyze the root cause of application errors

• Performance analysis: analyze response times and throughput

• Security audit: detect anomalous access patterns

• Incident response: investigate the root cause during an outage

Instructions

Step 1: Locate Log Files

Common log locations

/var/log/ # System logs /var/log/nginx/ # Nginx logs /var/log/apache2/ # Apache logs ./logs/ # Application logs

Step 2: Search for Error Patterns

Common error search:

Search ERROR-level logs

grep -i "error|exception|fail" application.log

Recent errors (last 100 lines)

tail -100 application.log | grep -i error

Errors with timestamps

grep -E "^[.*ERROR" application.log

HTTP error codes:

5xx server errors

grep -E "HTTP/[0-9.]+ 5[0-9]{2}" access.log

4xx client errors

grep -E "HTTP/[0-9.]+ 4[0-9]{2}" access.log

Specific error code

grep "HTTP/1.1" 500" access.log

Step 3: Pattern Analysis

Time-based analysis:

Error count by time window

grep -i error application.log | cut -d' ' -f1,2 | sort | uniq -c | sort -rn

Logs for a specific time window

grep "2025-01-05 14:" application.log

IP-based analysis:

Request count by IP

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

Activity for a specific IP

grep "192.168.1.100" access.log

Step 4: Performance Analysis

Response time analysis:

Extract response times from Nginx logs

awk '{print $NF}' access.log | sort -n | tail -20

Slow requests (>= 1 second)

awk '$NF > 1.0 {print $0}' access.log

Traffic volume analysis:

Requests per minute

awk '{print $4}' access.log | cut -d: -f1,2,3 | uniq -c

Requests per endpoint

awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20

Step 5: Security Analysis

Suspicious patterns:

SQL injection attempts

grep -iE "(union|select|insert|update|delete|drop).*--" access.log

XSS attempts

grep -iE "<script|javascript:|onerror=" access.log

Directory traversal

grep -E "../" access.log

Brute force attack

grep -E "POST.*/login" access.log | awk '{print $1}' | sort | uniq -c | sort -rn

Output format

Analysis report structure

Log analysis report

Summary

• Analysis window: YYYY-MM-DD HH:MM ~ YYYY-MM-DD HH:MM
• Total log lines: X,XXX
• Error count: XXX
• Warning count: XXX

Error analysis

Recommended actions

1. [Action 1]
2. [Action 2]

Best practices

• Set time range: clearly define the time window to analyze

• Save patterns: script common grep patterns

• Check context: review logs around the error too (-A , -B options)

• Log rotation: search compressed logs with zgrep as well

Constraints

Required Rules (MUST)

• Perform read-only operations only

• Mask sensitive information (passwords, tokens)

Prohibited (MUST NOT)

• Do not modify log files

• Do not expose sensitive information externally

References

• grep manual

• awk guide

• Log analysis best practices

Examples

Example 1: Basic usage

Example 2: Advanced usage