Pro Tips — Failover, Error Pages, and Debugging

Real-world expert tips for operating Nginx + Tomcat. This chapter covers fallback handling when upstream fails, custom error pages, health checks, and debugging integration issues.

Fallback Handling for Upstream Failures

Provide meaningful responses to users when Tomcat is down or returning errors.

proxy_next_upstream — Automatic Retry

upstream tomcat_backend {
    server 127.0.0.1:8080;
    server 127.0.0.1:8081 backup;     # Backup instance if main fails

    keepalive 32;
}

server {
    location / {
        proxy_pass http://tomcat_backend;

        # Retry next server on upstream failure
        proxy_next_upstream     error timeout http_502 http_503 http_504;
        proxy_next_upstream_tries       2;
        proxy_next_upstream_timeout     10s;
    }
}

Error Interception — Custom Error Pages

server {
    listen 443 ssl;
    server_name example.com;

    proxy_intercept_errors on;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
    }

    error_page 500 /error/500.html;
    error_page 502 /error/502.html;
    error_page 503 /error/503.html;
    error_page 504 /error/504.html;

    location /error/ {
        root /var/www/myapp;
        internal;
        add_header Cache-Control "no-store";
    }

    # JSON error response for API clients
    location /api/ {
        proxy_pass http://127.0.0.1:8080;
        error_page 500 502 503 504 @api_error;
    }

    location @api_error {
        default_type application/json;
        return 503 '{"error":"service_unavailable","message":"Service temporarily unavailable."}';
    }
}

Static Fallback Page

<!-- /var/www/myapp/error/502.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="refresh" content="10">
    <title>Service Unavailable</title>
    <style>
        body { font-family: sans-serif; text-align: center; padding: 50px; }
        h1 { color: #e74c3c; }
    </style>
</head>
<body>
    <h1>Service Temporarily Unavailable</h1>
    <p>The page will automatically refresh in 10 seconds.</p>
    <p>Contact: support@example.com</p>
</body>
</html>

Health Check Configuration

Tomcat Health Check Endpoint (Spring Boot)

# application.yml
management:
  endpoints:
    web:
      exposure:
        include: health
  endpoint:
    health:
      show-details: never

# External health check script
#!/bin/bash
HEALTH=$(curl -sf http://localhost:8080/actuator/health | grep -c '"status":"UP"')
if [ "$HEALTH" -eq 0 ]; then
    echo "Tomcat unhealthy — restarting"
    systemctl restart tomcat
fi

Internal Health Check Endpoint for Nginx

server {
    listen 8888;
    server_name localhost;
    allow 127.0.0.1;
    deny all;

    location /health {
        proxy_pass http://127.0.0.1:8080/actuator/health;
        proxy_connect_timeout 3s;
        proxy_read_timeout    5s;
        access_log off;
    }
}

Debugging — Issue-by-Issue Diagnosis

502 Bad Gateway

# 1. Check Tomcat port listening
ss -tlnp | grep 8080
# No result = Tomcat not running

# 2. Direct Tomcat access test
curl -v http://localhost:8080/
# No response = Tomcat problem

# 3. Check Nginx error log
tail -50 /var/log/nginx/error.log
# "connect() failed (111: Connection refused)" → Tomcat down
# "upstream timed out"                          → Tomcat slow response

504 Gateway Timeout

# Check Tomcat processing time (last field in access log: response time ms)
tail -100 /opt/tomcat/latest/logs/localhost_access_log*.txt | \
  awk '{print $NF}' | sort -n | tail -10

# Check and adjust Nginx read_timeout
grep "proxy_read_timeout" /etc/nginx/sites-available/myapp.conf

# Check Tomcat thread status
curl -u admin:password http://localhost:8080/manager/status

Connection Reset / Drops

# Check TIME_WAIT excess
ss -s | grep TIME-WAIT

# Check keepalive mismatch
# If Tomcat closes the connection before Nginx, Nginx returns 502

# Fix: Set Tomcat keepAliveTimeout higher than Nginx keepalive_timeout
# Nginx: keepalive_timeout 65s;
# Tomcat: keepAliveTimeout="70000" (70 seconds)

Debug Headers for Development

server {
    # Add debug headers to responses (development environment)
    add_header X-Upstream-Addr    $upstream_addr;
    add_header X-Upstream-Status  $upstream_status;
    add_header X-Upstream-Time    $upstream_response_time;
    add_header X-Request-ID       $request_id;

    location / {
        proxy_pass http://tomcat_backend;
        proxy_set_header X-Request-ID $request_id;
    }
}

Log Correlation — Tracing Nginx ↔ Tomcat

log_format trace '$remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 'req_id=$request_id '
                 'upstream_time=$upstream_response_time';

access_log /var/log/nginx/myapp_access.log trace;

// Spring Boot — include X-Request-ID in logs
@Component
public class RequestIdFilter implements Filter {
    @Override
    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        String requestId = ((HttpServletRequest) req).getHeader("X-Request-ID");
        MDC.put("requestId", requestId != null ? requestId : "none");
        try {
            chain.doFilter(req, res);
        } finally {
            MDC.clear();
        }
    }
}

Maintenance Mode

# touch /var/www/maintenance.flag  → enable maintenance mode
# rm /var/www/maintenance.flag     → disable maintenance mode

server {
    listen 443 ssl;
    server_name example.com;

    location / {
        if (-f /var/www/maintenance.flag) {
            return 503;
        }

        proxy_pass http://tomcat_backend;
    }

    error_page 503 @maintenance;
    location @maintenance {
        root /var/www/myapp;
        rewrite ^(.*)$ /error/maintenance.html break;
        add_header Cache-Control "no-store";
        add_header Retry-After 3600;
    }
}

Integration Health Check Script

#!/bin/bash
echo "=== Nginx + Tomcat Integration Checklist ==="

echo -n "Nginx config: "
nginx -t 2>&1 | grep -q "successful" && echo "✅ OK" || echo "❌ Error"

echo -n "Nginx running: "
systemctl is-active nginx > /dev/null && echo "✅ Running" || echo "❌ Stopped"

echo -n "Tomcat running: "
systemctl is-active tomcat > /dev/null && echo "✅ Running" || echo "❌ Stopped"

echo -n "Tomcat :8080: "
ss -tlnp | grep -q ":8080" && echo "✅ Listening" || echo "❌ Not listening"

HTTP_CODE=$(curl -sk -o /dev/null -w "%{http_code}" https://localhost/)
echo "HTTP status: $HTTP_CODE (expected 200)"

RESPONSE_TIME=$(curl -sk -o /dev/null -w "%{time_total}" https://localhost/)
echo "Response time: ${RESPONSE_TIME}s"

echo "=== Checklist complete ==="

Summary

Situation	Solution
Tomcat down	`proxy_next_upstream error timeout` + backup server
5xx error pages	`proxy_intercept_errors on` + `error_page`
502 occurring	Check Tomcat listening → direct access test → Nginx logs
504 occurring	Increase `proxy_read_timeout` → analyze Tomcat processing time
keepalive drops	Tomcat `keepAliveTimeout` > Nginx `keepalive_timeout`
Maintenance mode	Flag file + `if (-f)` + return 503

Fallback Handling for Upstream Failures​

proxy_next_upstream — Automatic Retry​

Error Interception — Custom Error Pages​

Static Fallback Page​

Health Check Configuration​

Tomcat Health Check Endpoint (Spring Boot)​

Internal Health Check Endpoint for Nginx​

Debugging — Issue-by-Issue Diagnosis​

502 Bad Gateway​

504 Gateway Timeout​

Connection Reset / Drops​

Debug Headers for Development​

Log Correlation — Tracing Nginx ↔ Tomcat​

Maintenance Mode​

Integration Health Check Script​

Summary​