keepcalculator

06 Numbers and Powers

Terminal Tools for Tolerable Log Viewing

Card No. 06·28109 · Updated · Standard formula, shown below

Yes, there are terminal tools that make working with logs genuinely tolerable, and the best ones turn a wall of text into a searchable, filterable, color-coded stream you can actually follow. This article covers the most useful ones, how they work, and how to choose for your workflow.

Why Logs Are Painful by Default

Plain log files are just lines of text, often with timestamps, levels, and messages. The problems are universal: too much noise, no structure, and no easy way to jump to the interesting part. When you have thousands of lines per minute, scrolling with less or tail becomes a guessing game. The tools below solve this by adding filtering, highlighting, and real-time navigation without leaving the terminal.

The Core Tools: lnav, GoAccess, and multitail

Three tools stand out for general log work:

  • lnav – parses log formats (syslog, Apache, Nginx, JSON, and more) into a structured view. It colorizes by severity, lets you filter with SQL-like queries, and even shows statistics like error rates over time. It’s the closest thing to a GUI log viewer in the terminal.
  • GoAccess – a real-time web log analyzer. It runs in the terminal and gives you a dashboard of visitors, requests, status codes, and referrers. It’s not a general log tool, but for Apache/Nginx access logs it’s unbeatable.
  • multitail – shows multiple log files side by side in one window, with per-file coloring and filtering. It’s simpler than lnav but great when you need to watch several services at once.

How They Work: Parsing and Streaming

Most of these tools read log files as they are written (like tail -f) but add a layer of interpretation. For example, lnav detects the log format by matching patterns (timestamps, IPs, log levels) and then builds an in-memory database. This allows you to query with SQL, jump to errors, and see a histogram of events over time. GoAccess parses access logs line by line, updates counts in memory, and renders a dynamic dashboard using ANSI escape codes to refresh the screen. multitail simply splits the terminal into regions and applies regex-based color rules to each line.

Choosing the Right Tool for Your Use Case

Your choice depends on what you’re doing:

  • Debugging a single application – lnav is the best all-rounder because it handles multiple formats and lets you search/filter interactively.
  • Monitoring web traffic – GoAccess gives you instant metrics without setting up a separate analytics service.
  • Watching multiple services – multitail keeps an eye on several logs at once.
  • Simple follow and search – if you just need to tail and grep, the built-in tail -f plus grep might be enough, but you lose color and context.

Common Problems and How to Solve Them

Even with these tools, you’ll hit issues:

  • Log rotation – when a file is rotated, tail -f stops following. Use tail -F (capital F) to follow by filename, and lnav/GoAccess handle rotation automatically.
  • Huge files – lnav can load files that are hundreds of MB, but it may be slow. Use lnav -n to skip the interactive UI and just print filtered results.
  • Non-standard formats – lnav lets you define custom formats in a config file, but it takes time. For quick hacks, use grep with context flags.
  • Color blindness – lnav’s default colors can be hard to distinguish; use --theme to switch to a high-contrast theme.

Comparison with GUI Log Viewers

Terminal tools are not always better than GUI ones. If you need complex multi-file correlation or a point-and-click interface, tools like Kibana (with Elasticsearch) or Graylog are more powerful. But for quick, on-the-server debugging, terminal tools win because they are lightweight, work over SSH, and have no browser overhead. They also integrate with other CLI tools (like awk and sed) for ad-hoc processing.

Cost and Availability

All three tools are open-source and free. lnav and multitail are available in most Linux package managers (e.g., apt install lnav multitail). GoAccess is also in repos, but you may need to build from source for the latest version. They run on macOS via Homebrew as well. There is no licensing cost, but you invest time in learning the keybindings and configuration.

Alternatives and Extras

If these don’t fit, consider:

  • jq – for JSON logs, pipe them through jq to pretty-print and filter.
  • ripgrep – a fast grep that respects .gitignore and can search recursively; pair with tail for live filtering.
  • lnav’s SQL – you can write queries like SELECT * FROM logline WHERE level='error' to get a subset.
  • Logwatch – a batch report generator that summarizes logs daily via email, useful for overnight analysis.

Frequently Asked Questions

Q: Can I use these tools on Windows?
A: lnav and multitail are Unix-based, but you can use them in WSL (Windows Subsystem for Linux). GoAccess also runs in WSL. Native Windows alternatives are limited; PowerShell’s Get-Content -Wait is a basic option.

Q: How do I filter logs for a specific time range?
A: In lnav, you can use :filter with a regex, or write an SQL query with a timestamp column. For a quick grep, use grep '2025-03-01 10:' file.log to match the hour.

Q: What’s the best way to follow a log file that’s being written very fast?
A: Use tail -F with --pid if you want to stop when the writing process exits. For high volume, consider using lnav with its built-in rate limiting, or pipe to grep to reduce noise.

Q: Can I make lnav remember my custom formats?
A: Yes, you can define formats in ~/.config/lnav/formats/ as JSON files. The documentation has examples for common formats like Rails or Docker.

Q: Is GoAccess only for web logs?
A: Yes, it’s specifically designed for Apache/Nginx access logs. For other logs, use lnav or multitail.

Q: How do I colorize logs without installing anything?
A: You can use grep --color=always with a pattern, or use ccze which is a lightweight colorizer that works with tail. But it doesn’t parse structure.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →