Secrets have to be somewhere. Environment variables are not a good place for them, but if you can't trust your filesystem to be secure, you're already screwed. There's no where else to go. The only remaining place is memory, and it's the same story.
If you can't trust memory isolation, you're screwed.
As a counterintuitive example from a former insider: virtually no one is storing secrets for financial software on an HSM. Almost no one does it, period.
There’s a whole class of security vulnerabilities that let you read from arbitrary files on the filesystem. So if you end up having of those vulnerabilities, and your secret is in a file, then the vulnerability lets the attacker read the secret. And on Linux, if you have such a vulnerability, you can use it to read /proc/PID/environ and get the environment variables, hence getting secrets in environment variables too.
However, the same isn’t necessarily true for memory. /proc/PID/mem isn’t an ordinary file, and naive approaches to reading it fail. You normally read a file starting at position 0; reading /proc/PID/mem requires first seeking to a mapped address (which you can get from /proc/PID/maps); if you just open the file and start reading it from the start, you’ll be trying to read the unmapped zero page, and you’ll get an IO error. Many (I suspect the majority) of arbitrary-file read vulnerabilities only let you read from the start of the file and won’t let you seek past the initial unreadable portion, so they won’t let you read /proc/PID/mem.
Additionally, there are hardening features to lock down access to /proc/PID/mem, such as kernel.yama.ptrace_scope, or prctl(PR_SET_DUMPABLE)-that kind of hardening can interfere with debugging, but one option is to leave it on most of the time and only temporarily disable it when you have an issue to diagnose
Also, memfd_secret supports allocating extra-special memory for secret storage, which the kernel can’t read, so it shouldn’t be accessible via /proc/PID/mem
This is maybe putting the cart before the horse a little bit. The reason there's a class of vulnerabilities that allow arbitrary read is that we've, as an industry, decided that we classify file access as a vulnerability. It's not that file access is somehow materially different or easier from any other security issue, it's just that we set that as one of the goals of an attack.
If you decide that an attack is successful when it reads a file, then you'll obviously get a clustering of successful attacks that read files.
Whereas, if all they manage to steal using a file read vulnerability is the code (possibly even just the binaries if you are using a compiled language like Go or Java) of your web app - that’s not good either, but it is a lot smaller headache. You’d much rather be having to tell the CEO “attackers stole the binaries of our app” than “attackers stole all the PII of our customers”. Both are bad but the second is a lot worse. The first kind of attack you possibly won’t be obliged to disclose, the second you legally will be
It strikes me that those envs might be particularly prone to corporate inertia, ieg "the current way passed security audit, don't change it or we need to requalify"
It's possibly also harder to rely on a HSM when your software is in a container? ( I'm guessing here tho )
It's irrelevant to me whether you believe it, it's happening today, and it happens with some of the top financial institutions and their subsidiaries and it's the same bureaucratic nonsense to move those teams to do something about it like it is anywhere else.
