Log Analysis

This guide covers log locations, formats, and analysis techniques for License Monitor and License Server Detail.

Log Locations

License Monitor

Log Type	Default Location	Configuration
Application	`/var/log/license-monitor/license_monitor.log`	`[daemon] log_file`
Systemd	`journalctl -u license-monitor`	-
Stdout (debug)	Console	`--debug` flag

License Server Detail

Log Type	Location	Notes
Application	Console/stdout	Next.js output
PM2 logs	`~/.pm2/logs/`	If using PM2
Docker logs	`docker logs container`	If containerized

Log Format

License Monitor Log Format

2024-01-15T10:30:45.123Z INFO [license_monitor::api] Request completed
  request_id=req-abc123 method=GET path=/api/health status=200 duration_ms=45

Components:

Timestamp (ISO 8601)
Level (ERROR, WARN, INFO, DEBUG, TRACE)
Module path
Message
Structured fields

JSON Log Format

When log_format = "json":

{
  "timestamp": "2024-01-15T10:30:45.123Z",
  "level": "INFO",
  "module": "license_monitor::api",
  "message": "Request completed",
  "request_id": "req-abc123",
  "method": "GET",
  "path": "/api/health",
  "status": 200,
  "duration_ms": 45
}

Log Levels

Level	Use	Examples
ERROR	Critical failures	Connection failures, crashes
WARN	Potential issues	High latency, deprecated features
INFO	Normal operations	Requests, status changes
DEBUG	Troubleshooting	Detailed flow, variable values
TRACE	Deep debugging	All function calls, data dumps

Common Log Patterns

Successful Request

2024-01-15T10:30:45.123Z INFO [license_monitor::api] Request started
  request_id=req-abc123 method=GET path=/api/licenses

2024-01-15T10:30:45.200Z INFO [license_monitor::api] Request completed
  request_id=req-abc123 status=200 duration_ms=77

Authentication Failure

2024-01-15T10:30:45.123Z WARN [license_monitor::auth] Authentication failed
  request_id=req-abc123 reason=invalid_api_key ip=192.168.1.100

License Server Error

2024-01-15T10:30:45.123Z ERROR [license_monitor::license] License query failed
  server=flexlm-01 error="Connection refused" attempt=3

Rate Limiting

2024-01-15T10:30:45.123Z WARN [license_monitor::rate_limit] Rate limit exceeded
  ip=192.168.1.100 limit=100 window=60s

Log Analysis Commands

Basic Filtering

# View recent logs
tail -f /var/log/license-monitor/license_monitor.log

# Filter by level
grep "ERROR\|WARN" /var/log/license-monitor/license_monitor.log

# Filter by date
grep "2024-01-15" /var/log/license-monitor/license_monitor.log

# Filter by request ID
grep "req-abc123" /var/log/license-monitor/license_monitor.log

Statistical Analysis

# Count errors by type
grep "ERROR" /var/log/license-monitor/license_monitor.log | \
    awk '{print $4}' | sort | uniq -c | sort -rn

# Count requests by status
grep "Request completed" /var/log/license-monitor/license_monitor.log | \
    grep -oP 'status=\d+' | sort | uniq -c

# Average response time
grep "duration_ms" /var/log/license-monitor/license_monitor.log | \
    grep -oP 'duration_ms=\d+' | \
    awk -F= '{sum+=$2; count++} END {print "Average:", sum/count, "ms"}'

Time-Based Analysis

# Requests per minute
grep "Request completed" /var/log/license-monitor/license_monitor.log | \
    cut -d'T' -f2 | cut -d':' -f1,2 | sort | uniq -c

# Errors in last hour
awk -v date="$(date -d '1 hour ago' '+%Y-%m-%dT%H')" '$0 ~ date' \
    /var/log/license-monitor/license_monitor.log | grep ERROR

JSON Log Processing

Using jq

# Parse JSON logs
cat /var/log/license-monitor/license_monitor.json | jq '.'

# Filter errors
cat /var/log/license-monitor/license_monitor.json | \
    jq 'select(.level == "ERROR")'

# Extract specific fields
cat /var/log/license-monitor/license_monitor.json | \
    jq '{timestamp, level, message}'

# Count by level
cat /var/log/license-monitor/license_monitor.json | \
    jq -s 'group_by(.level) | map({level: .[0].level, count: length})'

Streaming Analysis

# Real-time JSON log viewing
tail -f /var/log/license-monitor/license_monitor.json | jq '.'

# Real-time error filtering
tail -f /var/log/license-monitor/license_monitor.json | \
    jq 'select(.level == "ERROR") | {timestamp, message}'

Log Rotation

logrotate Configuration

/var/log/license-monitor/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 license-monitor license-monitor
    postrotate
        systemctl reload license-monitor > /dev/null 2>&1 || true
    endscript
}

Manual Rotation

# Rotate logs manually
logrotate -f /etc/logrotate.d/license-monitor

# Check rotation status
cat /var/lib/logrotate/status | grep license-monitor

Log Aggregation

Sending to Syslog

[daemon]
log_output = "syslog"
syslog_facility = "local0"

Sending to Elasticsearch

# Using Filebeat
filebeat.inputs:
  - type: log
    paths:
      - /var/log/license-monitor/*.log
    json.keys_under_root: true
    json.add_error_key: true

output.elasticsearch:
  hosts: ["localhost:9200"]
  index: "license-monitor-%{+yyyy.MM.dd}"

Troubleshooting with Logs

Finding Slow Requests

# Requests over 1 second
grep "duration_ms" /var/log/license-monitor/license_monitor.log | \
    awk -F'duration_ms=' '{if ($2 > 1000) print}' | \
    tail -20

Finding Failed Requests

# Failed requests (non-2xx status)
grep "Request completed" /var/log/license-monitor/license_monitor.log | \
    grep -v "status=2"

Correlation Analysis

# Find all logs for a specific request
REQUEST_ID="req-abc123"
grep "$REQUEST_ID" /var/log/license-monitor/license_monitor.log | \
    sort | less

Next Steps

Debugging Guide - Step-by-step debugging
Common Issues - Frequent problems
Performance - Performance analysis