Log Configuration and Analysis

Nginx logs are essential tools for diagnosing failures, analyzing performance, and conducting security audits. This chapter covers log format customization, level settings, and log rotation — everything you need in production.

---

Log Types

Log	Default Path	Purpose
access_log	/var/log/nginx/access.log	HTTP request records (IP, URL, status code, response time, etc.)
error_log	/var/log/nginx/error.log	Error and warning records (config errors, missing files, connection failures)

---

access_log Format Customization

Default Format

http {
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;
}

Custom Format with Response Time

log_format detailed '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" '
                    'rt=$request_time '
                    'uct=$upstream_connect_time '
                    'uht=$upstream_header_time '
                    'urt=$upstream_response_time';

access_log /var/log/nginx/access.log detailed;

JSON Format (for Log Aggregation Systems)

log_format json_combined escape=json
    '{'
        '"time_local":"$time_local",'
        '"remote_addr":"$remote_addr",'
        '"method":"$request_method",'
        '"uri":"$request_uri",'
        '"status":$status,'
        '"bytes_sent":$body_bytes_sent,'
        '"request_time":$request_time,'
        '"upstream_response_time":"$upstream_response_time",'
        '"http_referer":"$http_referer",'
        '"http_user_agent":"$http_user_agent",'
        '"http_x_forwarded_for":"$http_x_forwarded_for"'
    '}';

access_log /var/log/nginx/access.json json_combined;

---

Key Log Variables

$remote_addr             # Client IP address
$request                 # Full request line (method + URI + protocol)
$status                  # HTTP response status code
$body_bytes_sent         # Response body size (bytes)
$request_time            # Total request processing time (seconds)
$upstream_addr           # Upstream server address
$upstream_response_time  # Upstream total response time (seconds)
$upstream_connect_time   # Time to connect to upstream
$upstream_header_time    # Time to receive upstream headers
$http_referer            # Referer header
$http_user_agent         # User-Agent header
$scheme                  # http or https
$ssl_protocol            # TLS version (for HTTPS)

---

error_log Level Settings

error_log /var/log/nginx/error.log warn;

# Levels (more detailed → more messages):
# debug   → All debug info (very verbose, do not use in production)
# info    → Informational messages
# notice  → General notices
# warn    → Warnings (recommended for production)
# error   → Errors
# crit    → Critical errors
# alert   → Immediate action required
# emerg   → System is unusable

---

Disabling Unnecessary Logs

Exclude static file requests to reduce log volume:

location = /favicon.ico {
    log_not_found off;
    access_log off;
}

location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
    access_log off;
    expires 7d;
}

location /health {
    access_log off;
    return 200 "OK";
}

---

Log Rotation (logrotate)

/etc/logrotate.d/nginx

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    sharedscripts
    postrotate
        if [ -f /var/run/nginx.pid ]; then
            kill -USR1 `cat /var/run/nginx.pid`
        fi
    endscript
}

# Test rotation immediately
sudo logrotate -f /etc/logrotate.d/nginx

---

Practical Log Analysis Commands

Status Code Distribution

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

Top 10 IPs by Request Count

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

Top 404 URLs

awk '($9=="404") {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

---

Summary

Item	Setting	Recommendation
Format	log_format	Include $request_time for performance analysis
Error level	error_log	Production: warn, debugging: debug (temporary)
Static files	access_log off	Exclude favicon, images, etc.
Rotation	logrotate	daily + 14 days + compress
Analysis	awk, grep	Aggregate by status code, response time, IP