How did Nosey Parker Explorer come about? 2-3 years ago I was working on training ML models for tasks related to hardcoded secrets, such as TP/FP classification and detection. These are pretty specific tasks and there are no open-access datasets. So, I had to build my own dataset of several thousand labeled examples.
None of the existing tools for data labeling that I ran across was a good fit for the task. They were all either too general-purpose, too complicated to set up, or too slow. So I built my own proof-of-concept purpose-built terminal-based labeling app using the excellent Textual TUI framework [2] and DuckDB [3] to build fast faceted search. A couple weeks later I had Nosey Parker Explorer.
Nosey Parker Explorer proved very effective at the dataset construction task. My team of a few people used it to label about 15k examples collected from 2TB of input.
However, beyond dataset creation, Nosey Parker Explorer was a huge boon for security engineers on engagements. Once you are dealing with more than a few dozen possible findings, it is _hugely_ helpful to be able to interactively slice-and-dice the data. The largest-scale use of it that I'm aware of was an assumed-breach engagement where we had tens of thousands of potential findings from 20TB of scanned inputs.
Nosey Parker Explorer was far too useful to be left as a proof-of-concept or only used internally. A couple weeks ago I released it under the Apache 2 license.
If you want to try it, It's a Python application (not yet on PyPI). There are prebuilt zipapp releases, but you can also clone the repository and `pip install`. You will first need to use Nosey Parker to scan something. See the project's README for details.
Happy to answer questions.
[1] Nosey Parker: https://github.com/praetorian-inc/noseyparker
[2] Textual: https://textual.textualize.io
[3] DuckDB: https://duckdb.org
No comments yet.
