Show HN: PII-Shield – Log Sanitization Sidecar with JSON Integrity (Go, Entropy) (opens in new tab)

(github.com)

20 pointsaragoss3mo ago11 comments

What PII-Shield does: It's a K8s sidecar (or CLI tool) that pipes application logs, detects secrets using Shannon entropy (catching unknown keys like "sk-live-..." without predefined patterns), and redacts them deterministically using HMAC.

Why deterministic? So that "pass123" always hashes to the same "[HIDDEN:a1b2c]", allowing QA/Devs to correlate errors without seeing the raw data.

Key features: 1. JSON Integrity: It parses JSON, sanitizes values, and rebuilds it. It guarantees valid JSON output for your SIEM (ELK/Datadog). 2. Entropy Detection: Uses context-aware entropy analysis to catch high-randomness strings. 3. Fail-Open: Designed as a transparent pipe wrapper to preserve app uptime.

The project is open-source (Apache 2.0).

Repo: https://github.com/aragossa/pii-shield Docs: https://pii-shield.gitbook.io/docs/

I'd love your feedback on the entropy/threshold logic!

11 comments

aragossOP3mo ago

Update: Seeing some folks pulling the Docker image. Just a heads up — the default entropy threshold is 3.8, which is tuned for API keys. If you are testing with simple words like 'test', it might not catch them (by design). Check the README for tweaking PII_ENTROPY_THRESHOLD.

maxbond3mo ago

Cool project!

- Wouldn't this censor UUIDs? I want UUIDs to remain in my logs.

- The never "PII Shield" makes me think this would censor entities like names or social security numbers, rather than secrets. Not a big deal though.

aragossOP3mo ago

Thanks!

UUIDs: By default—no. Since UUIDs are Hex (limited charset 0-f), they have lower entropy than Base64 secrets. The threshold is tuned to sit safely above UUIDs but below API keys.

Naming: You are totally right. Currently, it focuses on "high-entropy PII" (passwords, auth tokens, session IDs) rather than names or SSNs. "Secrets Shield" might have been more precise, but naming is hard :)

hangonhn3mo ago

So depending on the context UUID can be PII. Is this something we can customize or adjust?

aragossOP3mo ago

Yes, absolutely.

You can fine-tune the sensitivity via the PII_ENTROPY_THRESHOLD environment variable.

If you consider UUIDs to be sensitive in your context (or if you are getting false positives), you can adjust the threshold. For example, standard UUIDs have lower entropy density than API keys, so slightly tuning the value (e.g. from 3.8 to 3.2 or vice-versa) allows you to draw the line where you need it.

1 more reply

gritspants3mo ago

Thanks for this. Showed some of my colleagues who thought it was cool also.

aragossOP3mo ago

Thank you! Hearing that it's being shared is the best kind of validation.

If your team ends up trying it out and has any feature requests (or catches any edge cases), I’d love to hear them!

johnthescott3mo ago

bravo. you are off to a good start.

aragossOP3mo ago

Thank you! Appreciate the kind words.

j / k navigate · click thread line to collapse

11 comments

aragossOP3mo ago

maxbond3mo ago

Cool project!

- Wouldn't this censor UUIDs? I want UUIDs to remain in my logs.

- The never "PII Shield" makes me think this would censor entities like names or social security numbers, rather than secrets. Not a big deal though.

aragossOP3mo ago

Thanks!

UUIDs: By default—no. Since UUIDs are Hex (limited charset 0-f), they have lower entropy than Base64 secrets. The threshold is tuned to sit safely above UUIDs but below API keys.

hangonhn3mo ago

So depending on the context UUID can be PII. Is this something we can customize or adjust?

aragossOP3mo ago

Yes, absolutely.

You can fine-tune the sensitivity via the PII_ENTROPY_THRESHOLD environment variable.

1 more reply

gritspants3mo ago

Thanks for this. Showed some of my colleagues who thought it was cool also.

aragossOP3mo ago

Thank you! Hearing that it's being shared is the best kind of validation.

If your team ends up trying it out and has any feature requests (or catches any edge cases), I’d love to hear them!

johnthescott3mo ago

bravo. you are off to a good start.

aragossOP3mo ago

Thank you! Appreciate the kind words.

j / k navigate · click thread line to collapse