dloss

I'd like to share "klp", a command-line log viewer for investigating small to medium-sized log files (up to ~100k lines). While there are faster and more mature log viewers out there, I think klp has some features that might be interesting:

- Visualize log level patterns with --levelmap showing the density of different log levels over time (e.g., IIIWWEEE for a sequence of INFO, WARNING, ERROR)

- Visual separation of time gaps with --mark-gaps INTERVAL (e.g., 1h) to spot suspicious quiet periods

- Condense bursts of events with --fuse INTERVAL (e.g., 1h) to highlight when activity starts and ends

- Built-in regexes for common patterns (URLs, IPs, file paths, email addresses, common error messages, and many others)

- Python expressions for complex filtering: --where "int(status) >= 500 and 'timeout' in message.lower()"

- Built-in support for logfmt, JSON Lines, CSV, SQLite and others, plus ability to write ad-hoc parsers in Python on the command line

- Custom output formatting using Python code and templates

klp is a single Python file with no dependencies beyond Python 3.7+. Install with pip install klp-logviewer or download the file from GitHub:

<https://github.com/dloss/klp>

I originally built this for my own debugging needs at work where I couldn't install other tools. When my employer allowed me to open-source it (MIT license), I added more features to explore ideas I hadn't seen in other log tools.

Since I'm just a single developer maintaining this in my spare time, I've included a carefully curated list of more robust alternatives in the README.

I welcome your feedback on klp and its features, and would be delighted if these ideas inspire other log tools - especially approaches like visual time gaps, fusing events and log level maps.

Hello HN community,

I’m currently developing a log parsing and log viewing tool (https://github.com/dloss/klp) and am in need of realistic log file examples that I can include in my documentation and use for internal testing. Specifically, I’m looking for log files that are:

1. Publicly available and permissible for redistribution under an Open Source license (ideally MIT license).

2. Structured logs, either in JSONL (JSON Lines) or logfmt format.

The purpose is to show how my tool can be used to effectively handle real-world data, and to identify new features that would be useful.

If anyone knows of datasets, repositories, or sources where I could find such log files, your guidance would be immensely helpful.

Thank you in advance for your assistance and suggestions!