So many people are reporting these in tons of different projects: https://github.com/search?q=regex+denial+of+service&type=iss...
Anyhow it is just annoying and they broke NPM Audit based on these reports.
It is good to fix all possible bugs, but many of these are not anywhere close to the level of bad that the reports are making them to be.
But maybe this is needed to just get rid of these issues in genera? So a wave of regex vulnerability reports and then we build this type of checking into prettier or similar and we do not have these in the future?
EDIT: It appears there as a project that found 100s of CVE reported Regex vulnerabilities in npm projects -- this is maybe one of the sources of mass reports. See the bottom of this resume: https://yetingli.github.io
They seem pretty adamant on filing CVEs despite what the owner says (It's normally fine but these DoS vulns require very large input to be handed into the function by untrusted sources, which given how these libraries work isn't going to be very common).
Now, I have people yelling at me about dependent packages not being updated because they don't understand version ranges, or because some audit states they are high vulns, or whatever.
Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say.
Because this isn't true. Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. These Regex issues should be fixed, libraries should update to safe versions, things should advance and any incentive we have we should use to make this happen.
But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a "critical" bug in Express, potentially allowing my server to be compromised.
I'm not really sure what the solution is here; NPM's just a package manager and doesn't know how you're using a given package. A simple heuristic distinguishing development dependencies and runtime dependencies in NPM Audit might be a start, but that doesn't help with situations like create-react-app's react-scripts where everything, runtime or dev dependency, is a transitive dependency of one package declared as a runtime dependency.
A “Critical” bug in a dev context should mean something very different from a “Critical” bug in a prod context. A “Critical” devDependency bug should be either a direct threat to the developer’s context, either by infecting the dev machine or by injecting a supply-chain problem, worming it’s way into downstream contexts.
npm audit is just not granular OR careful enough to address these issues appropriately.
For example, a RegEx DDoS vulnerability in Express would show up as high severity, while the same would not show in the bundler you use, or any package that your bundler has in its dependency tree.
Accepting regexes from user input is a really insidious class of bug that can go undetected for years. I've seen real outages caused by it, so it's absolutely worth doing something proactive about.
But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?
Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast...
Edit: LWN mention https://lwn.net/Articles/801157/
Spender has a much more nuanced, informed view. I think it covers the issues of the CVE process well, but doesn't make the same mistakes that Greg does.
https://www.grsecurity.com/reports_of_cves_death_greatly_exa...
I think regexes are often used as a quick and dirty solution to problems, which should be solved differently. But once the regex "works" and is in place, others begin to rely on that output. Over time cruft begins to accumulate and the regex is forgotten or at least never replaced with anything more appropriate.
Surprise: The most common parser combinator libraries do backtracking. That's exactly the problem. Any solution as widely used (if not overused) as regular expressions ends up exposing a number of dark corners where the design isn't as clean and tight as you would want it. There are lots of better ways, but most of them are specialized and are totally unsuited for significant areas where people need something.
That said: yes I've used LR(1) parsing (not LALR) using a library that uses parser combinators with a good interface, and it's more powerful than regex and worth it for the right usecase.
Good news (well, probably). JavaScript (ECMAScript 2018+) now supports named matching groups.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
If people are competing for CVEs, then why not work out a way to better differentiate them them through scoring and make this visible. The goal would be for attention to shift to the scoring instead of only a CVE count. Offer both views of the world, so tools could still fall back on the problematic listings they get today.
Apply machine learning to classify CVEs based on the reputation of the reporter, blast radius, or other criteria. Use that to drive community review and scoring.
I would not see this a panacea because it brings a lot of challenges (a la StackOverflow), but it would be much better than what we have today.
I suspect it's an impossible challenge, but I only dip into this domain casually so maybe someone has better ideas.
I'm not a react developer, I was experimenting with it for a new project. I finished the tic-tac-toe tutorial, then tried to throw bootstrap on top to build from there. It told me there was 97 vulnerabilities (85 moderate, 12 high)...
I just deleted the directory and went back to vanilla JS. This is a fun side project, I don't need that.
My tweet about it: https://twitter.com/preinheimer/status/1402785757962592256
However a beginner that doesn't know what impact they have of course is scared if when installing a library it tells you that there are all that vulnerabilities.
After all, npm can execute any script with the users permissions on install…except often (compared to bash) it’s less easily inspected due to the common use of nested dependencies!
I, too, would delete my node_modules, and if I even wanted to move forward at that point, would probably waste at least half a day looking up the Critical vulns and discovering that they are probably not at all critical in my particular scenario. Like not at all for the 99.99% use case.
After experiencing something like that, it’s just like the article says. “The boy who called wolf.” Really terrible use of the labels “Critical” and “High”. The labels are fine, but the way they are applied is just stupid.
Next up, npm link: broken by design
Synopsis of the chapter: A command with broken behavior that has been reported since as early as 2015, but that "got lost" every time the winds changed and the project decided to change where to manage bugs. What will happen in the latest attempt from an affected user? Tune in and be ready for an exciting ride!
https://github.com/npm/cli/issues/2372
Spoiler: bugs are not sentient beings that solve themselves just by closing the issue (or the whole issue tracker, for that matter).
EDIT to clarify: Sorry for the snarkyness. I just find it funny in a sarcastic way that up until I reported the issue in 2020, the issue had been reported repeatedly but "lost" in the way because the project closed or ignored the issue every time it changed issue tracker. Which happened twice since 2017! so go figure the amount of reports that had gone to waste. On the flip side, this time they haven't changed platforms (yet), although the issue has been closed prematurely anyway.
I would like to wait to see if the rearchitecture for npm 7 actually allows them to test for regressions more productively, but at this point I don't know if I have the stamina to wait for my company to migrate to node 16.
Someone offers me a job doing Elixir or non-webapp stuff and I'm out. Probably permanently.
And when you inlined a version that later really does have a vulnerability, it is not easily flagged or fixed by your consumers.
The tension between "upgrades (especially of indirect dependencies) might break" and "upgrades (especially of indirect dependencies) might be necessary to fix bugs or patch security vulnerabilities" is real. There is no magic bullet to get around it. There are practices to try to balance it -- which generally involve ecosystem-wide commitment to backwards compatibility, reflected in semantic versioning (and minimizing major releases).
That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem. It makes the dependency tree even more insane, and it also accomodates a false belief that it's fine to lock to very specific versions of dependencies -- because it won't prevent other dependencies from co-existing in tree with other conflicting requirements after all. Which then accomodates a false belief that dependency maintainers don't need to worry too much about backwards compatibility, after all consumers can just lock to specific working versions... and now we wind up with insane dependency trees which depend on vulnerable versions in ways that require a whole bunch of releases to resolve.
I think a big part of it is that due to much stronger pressure on bundle size than most other environments, each library tends to be small, so there have to be more to carry the same amount of functionality.
Duplicates are certainly a contributing factor as well, and small bundles compound with allowed-duplication to further increase the tree size. I think that small package size also probably makes it harder to require a single version for each dep, since there are going to be more edges in the graph and therefore more relations for library maintainers to keep track of (including what would in other languages be intra-package requirements), so you're more likely to get version incompatibilities.
Aren't npm dependency trees so large because JavaScript doesn't have much of a standard library? And also, similarly, the community is so large and has been moving so fast that even de facto standards have difficulty forming and surviving at a large scale across the community.
Add to that everything else—the fast pace of changes, javascript "culture", the weak standard library, the tendency to patch in what ought to either be basic language features or else avoided in favor of more-vanilla idioms, often in competing and incompatible ways—and all that is how you end up with 20 slightly-different copies of the same damn library in your dependency tree, and then 20 other copies of another library that does the same thing.
I would add that I think the devDependencies solution is underrated. Not using it is a bad practice. npm has a nice feature, `npm prune --production` which will remove all the dev dependencies for you resulting in a clean build of the program. You can easily have things set up so that none of the development dependencies that have all these audit issues are ever present on your production machines if you do things right.
Furthermore, if you have a project with end users, devDependencies allows you to make clear in your issue template that audit issues that don't show up in a production build are very likely to be false positives and will probably be closed without comment. If you aren't properly isolating your dependencies, you can't take advantage of this.
At the end of the day, as you say, the issue is largely with Node projects having too many dependencies and a large number of relatively minor issues that affect very specific use cases get reported. That's a hard ecosystem problem to solve, not necessarily something that indicates an inherent problem with npm audit; but if someone isn't using devDependencies, that alone could constitute an enormous improvement in their workflow.
However it's incompatible with optionalPackages, so I'm carrying around some tertiary dependencies that are a small but noticeable fraction of the entire archive size.
For my hundreds of repos (Java, Scala, JS, Typescript, Python...), Snyk flags 99% of the CVEs for the JS repos. Shocking how I've only seen a few dozen or so Java based CVEs flagged over the last few years.
Perhaps it's because my NPM based repos have ~10K more dependencies? That and the Java stdlib handling most needs w/ the vanilla lang.
Some of us are considerably salty about it. Especially the design-by-PR aspects of the whole thing that have resulted in confusing gyrations from one version to another.
https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabili...
We disable [1] audit entirely because it's not a good default behavior within a monorepo. It spams the hundreds of developers with the list of "vulnerabilities" on every install, but only a few folks should really be upgrading packages.
We then run audit in non-blocking CI and track the total number of issues and mostly focus on critical ones.
For vulnerabilities that we determined weren't an issue ever (vuln in frontend framework we didn't use), or weren't high priority enough to P0 through, we needed some way to ignore either permanently or temporarily specific vulnerabilities.
Given the enormous dependency sets eg react create, you'd think the tools would be better at managing them.
I can't believe no one here has mentioned the fantastic tool "better-npm-audit" which can be included as an npm dependency[0] and lets you add specific vulnerabilities to an ignore list.
The ignore list is actually a JSON config file stored alongside package.json in the repo, so only one developer ever needs to see the npm audit warning and can mute it for everyone else (after getting their PR approved).
Even better, the config file lets you specify an expiry date for each entry in the ignore list, and provide a note, such as a link to the upstream issue being worked on, so that you can periodically be reminded to go back and check if a new version is available which can give more confidence that your code really isn't affected.
I think that developers might have to be instructed to use the "--no-audit" option to "npm install" if they don't want to see the (false positive) warnings that the default behaviour produces, and that's a bad habit to learn if not all projects they work on are using "better-npm-audit". I don't know if there is a way to make that option the default on a per-project basis.
From my experience most audit flags happen because a new vulnerability is discovered, which means stopping a deploy doesn't actually do anything helpful.
Or, in other words, it's not that insanely complicated, because they have tools that make it look a lot like multiple repositories. And the large companies using multiple repositories have tools that make them look just like those monoreps. And HN has all those interesting threads full of people saying why one is better than the other.
We come from separate repos on past versions of the code and the monorepo setup has sped dev up dramatically, even for a team of 2. Having the docs next to our code and autogen some of it from the source code, all is the same repo is just one advantage.
It's a small project but you can view the setup here: https://github.com/lowdefy/lowdefy
You can read a bit about the approach’s benefits and drawbacks from someone at Google here: https://medium.com/@Jakeherringbone/you-too-can-love-the-mon...
While complicated, in my experience the tooling to get the same productivity from hundreds of separate repos is at least as complicated and generally discourages folks from contributing to codebases outside their own.
Our monorepo encourages a collective velocity culture where everyone is pushing every project forward. If you upgrade React or Node, fix a security issue, or implement an optimization, the hundreds of apps are all improved at the same time. It's harder for one team to "leap forward" in terms of tech stack quickly, but at the same time it's far less likely we end up with hundreds of outdated or abandoned codebases since everyone is collectively improving the repo together.
For example I've rarely seen a single engineer upgrade a thousand separate projects across a thousand git repos with complex nested dependencies very successfully, but that happens every day in our monorepo.
https://research.google/pubs/pub45424/
That said (1) “you are not Google” as the saying goes (2) Google built their VCS from scratch in house for scale.
However it’s not tops hard to do this with just git and some rules/patterns for organization.
Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities.
If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense.
I do agree with the author's note about providing a better way to provide feedback on severity reviews.
npm audit is better than no npm audit...telling people it's broken by design is going to discourage them from using it completely. smh.
A big part of the problem is there is no reliably way to "just upgrade it" today in npm:
- `npm audit fix --force`, which is supposed to do that, is buggy and doesn't work
- There is no way to override a transitive dependency with npm (there is with Yarn though, so hopefully this feature will come to npm soon)
- Sometimes the fix in transitive dependency _also_ includes breaking changes (e.g. because it wasn't backported), and so updating it subtly breaks the logic
>Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea
I don't think databases can do it, but what I'd like to be able to do is to be able to provide advisory that the way _my package_ uses a concrete transitive dependency is not affected by that vulnerability. Because as the package owner I _do_ have that context. I understand there may be significant issues with this approach though!
The last Create React App I worked on (~3 months ago) had over 500 "vulnerabilities" reported by npm/yarn audit. Most of the reported vulnerabilities were obviously junk. As the author noted, there's no need to report vulnerabilities in the same dependency in every path through the dependency graph. The noise made it very difficult to sift through the output for anything useful. Even then, I have my doubts about how applicable the results are because with tree shaking of an SPA, it's quite possible the vulnerable part of a dependency is never even used.
Upgrading a dependency can go anywhere from trivial to absolute nightmare. Usually somewhere in the middle where it takes time and effort to do right. A typical JS app nowadays has hundreds if not thousands of dependencies. I'd love to see a world where "just upgrade" is reasonable advice, but we are not there.
If I wanted shitty false alarms about bogus security issues, I'd get a PCI-DSS audit.
Imagine if you had 3 days to fix the regex DoS issue shown there, screw your release freeze and your current sprint plans, and you have the real working environment in some companies.
I've also heard reports of people trying to claim bug bounties for similar reports, or security vendors that run automated tools that detect for issues of similar (lack of) value.
CVEs try to supplement with flags for remotely exploitable, etc. but it still intentionally leaves a lot of space for interpretation, which is necessary for any normal enterprise.
The problem comes in when analysts (or their managers) interpret inflexibly, without appropriate technical context, or without understanding business impact and tradeoffs.
If you look at the workflow, it is hard to close the loop from engineering or IT back to security. We need a set of controls for secops departments' output relevance and departmental interoperation.
One if them is scanning all your dependencies, and its so frustrating. Because that team obviously has no idea what any of the dependencies do or how they’re being used. All they see is a red flag, and tell you to fix it. Good luck when they tell you this days before a release, and a week after the code is frozen. They’ll just block your release without a second thought.
Funnily, in our last release, some of our NPM packages were flagged as a risk, obviously without explanation. The thing was, these packages where dependencies of another package. Obviously we can’t go around updating open source code, just because the security team in our company told us.
This isn't obvious to me. Most open source projects accept contribution from others.
So the article boils down to “a bunch of these vulnerabilities aren’t applicable to my app which is built using a specific NPM package”.
Congratulations. Welcome to the world of practical information security.
As a security engineer, we’re lucky if your favorite package manager even associates vulnerability information with your packages. Never mind that you’re pulling in code at build time from who-only-knows-where that almost certainly wasn’t security reviewed. But that’s for another post.
Now you have a package manager that is kind enough to tell you that there might be a vulnerability, and you’re upset because NPM did not have specific logic to understand the mechanics of one of the packages it manages? And the upshot is that you have to apply judgment and attention to each notification? Is that a tear in my eye- no, wait, it’s just an eyelash.
How many packages are there? I’m sure the NPM guys have nothing better to do than to build context awareness for every package in their repository.
In all seriousness, I would love to see context awareness in vulnerability reporting. But expecting a package manager to understand that because of your specific choice of framework, that the DoS could only be conducted by an admin of your app, seems unreasonable to me.
People throw around “false positive” as a catch-all for “I don’t care about this”. But there are a number of distinct reasons one might not care:
- the scanner is wrong (e.g. there’s a code bug in the scanner like detecting “printed” instead of “sprintf”.
- the output is wrong because the vulnerability isn’t a vulnerability anytime, anywhere under any circumstances
- the scanner is correct, but environment or mitigation’s mean it doesn’t apply to me or the severity is wrong in my environment (this is the case here)
- the scanner is correct, but is giving me output I don’t care about (eg I want to filter for only high/critical but I can’t)
- there is so much output that I can’t pay attention to all of it; it’s so overwhelming that I can’t stand to look at it
Many security products have problems with output that is too verbose. This seems like a trivial problem to work around here; after you’ve triaged that a particular vulnerability doesn’t apply to a particular project, then filter it out with grep -v (our put a bunch of such lines in a bash script and always pipe npm audit output to the script.
Also, I sympathize with concerns that the vulnerability reporter perhaps scored the vulnerability too high. But there’s no perfect solution for that, and I’d rather be aware of a vuln and choose to ignore it, than not be aware at all.
I'm not even sure if I'd classify it as a security bug at this point, I'd just classify these examples as configuration options to be aware of. Document and ignore.
With the explosion in Javascript packages to implement trivial behaviours, the NPM dependency hell and all the other cruft that "modern" frontend development requires, there are more important security issues to monitor.
Many real issues exist, but their classification is ridiculously flawed. Every security researcher tries to mark their vulnerability up to be the next Eternal Blue of Javascript development, but these vulnerabilities rarely matter.
I think a much bigger problem from a security standpoint is not necessarily the quality of these bug reports, but the sheer number of dependencies even a basic React project has these days. Supply chain attacks are real and the javascript world can do with some dependency purging. Any of the 200 single-line Javascript libraries can be compromised at any point and infect developers all over the world the next time they update their dependencies.
The way people "just" seem to add new dependencies to projects terrifies me. left-pad hit the frontend world and changed nothing. twilio-npm infected developer machines, and create-react-app imports over 1500 libraries as if it's the most normal thing in the world. It's absolute madness.
I have no idea how to fix all of this. It's only a matter of time before someone replicates the research people have done in uploading packages with typos in the name (which have reached into big, famous companies like Apple and Facebook) and start doing some serious damage. Maybe it's already happening: a package having been taken over for scraps, the owners lying in wait while everyone downloads and updates their React/Svelte/etc. packages, ready with their cryptolockers to strike frontend devs and build servers the world over with the push of a button. I wouldn't be surprised, not in the slightest.
Ideally you'd want to show only relevant alerts, but... how? You'd need to know which kind of errors are relevant for a particular project, but that'd require solving the halting problem. This is made much worse by that it's JS.
Some libraries have an enormous complexity and attack surface. Take a database interface -- there probably is a vulnerability in some obscure corner the typical person may not even know exists.
I think though at the very least some improvement could be made by better priorization and categorization. DoS by exploiting a regex parser isn't that big of a deal if your project is just getting started, but an exploit allowing arbitrary code execution would still be.
In Dan's twitter thread, he calls this out as a viable solution.
It seems like a lot of this has been designed for Node (backend) development, whilst ignoring the fact that NPM is probably used more heavily for front-end development at this point.
In highly regulated industries, shipping code flagged as having a vuln without a manual approval could be a liability.
This wrapper around npm takes an allowlist argument, and our procedure is for an engineer to review the failing build, determine if the vulnerability (ugh, usually regex ddos or prototype pollution) is present in code that runs only at build time with trusted inputs, only on the client which is by definition untrusted, or in our webserver which takes in untrusted input.
As long as it's either of the first two, we document it in a commit and comment and redeploy. It's annoying, but it's far better than npm audit forcing a fix.
Let me plug this as it contains a lot of references https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Meanwhile I'll try to get someone from IBM involved in the OpenJSF collab space
A vulnerability is a vulnerability, whether it applies to your context or not. A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?
- If your dependencies have security vulnerabilities, apply the updates.
- If you cannot update because there's no fix available, let your org or you assess the risk and go from there.
- If you cannot update because it breaks your app, {find a replacement, fix it yourself, let your org or you assess the risk and from from there}.
A sensible org has a process that freezes releases until known security issues are fixed. Freezes can also be opposed by devs and are evaluated on a case-by-case basis (because sometimes they are not relevant to the product, or someone steps up to take the blame for incidents and the org agrees).
We might not like it because it disturbs the "flow", but it's just part of the engineering process. More to the point though, why not take this opportunity to teach newcomers how to code properly, pick well-engineered and -written programs, and handle this vulnerability management process altogether? In any case, I hope newcoming-dev is not going to push to prod anytime soon. ;)
edit: formatting
I think a better metaphor might be, the button on your key fob that opens the trunk doesn't work, so you have to open the trunk using a physical lever. Every time you start the car, a loud warning siren sounds, and a red message appears on the dashboard to tell you that there is a "high impact" problem with your car, and you need to take it to be serviced. If you were merely the owner of the car, and other people also had to drive it, you might understandably be the target of several complaints about this "high impact" problem.
Ever seen apps which are basically impossible to patch because devs have ignored patching for so long that it’s basically impossible to version bump things sanely? I have.
For projects you own you'd have to flag each dependency path though, because for example, one dependency may not have the input for the regex exposed to the end user, while another dependency could.
Maintainers of libraries should also flag the security issues, and an issue with these two statuses on them wouldn't be raised by default. Options should be available to list them though for auditing.
For more security critical teams/projects, a per project setting to alert about any issues the maintainers have flagged irrelevant or mitigated and you'd have to accept them before it it would stop alerting about them.
Is yarn the better option? What is our path forward?
It simply don’t happen in Python, Go or Rust (or even Java) because the languages comes with a rich standard library. Javascript comes with just the basics, everything else is a dependency. It’s not uncommon for people to audit their dependencies in Python or Go, but you pull in maybe 10 or 20. A basic Javascript app easily pull in 100 times that, how are you suppose to deal with that?
This has not been my experience at all. Most Rust developers unfortunately seem quite happy to adopt modern programming bad practices. (There are many exceptions, of course.)
Example: I went with the first Rust program I could find installed on my computer, tealdeer. It's a dead simple program: run `tldr progname` and it will print a handful of examples of how to run `progname` in your terminal. Run `tldr --update` and it will download the latest version of the database containing these examples from a web server.
To build this extremely basic CLI program, I need ONE HUNDRED AND NINETEEN distinct crates.
Java has rich standard library. Some would say - too rich. But anyway - very few people would need customized collections, standard library covers all the needs. And where it does not cover all the needs, there are few commonly accepted additions like apache-commons or google guava. So that one solves `leftpad`-like issues.
Another difference is that Java is old. Most needs were covered by many libraries and few libraries survived which are good enough. It's again some commonly accepted wisdom, so you don't really need to search for many options. You have one good enough option that you'll use and move on.
So it's not about maven/gradle vs npm, it's more about ecosystems. I don't think that porting maven to JS world would change anything.
[1]: https://pnpm.io/
There are still problems:
- The decisions file gets unweildy, mainly because every time it fixes something it writes to the file. You probably only care about ignores. It's also append only, though you could manually clear it down sometimes.
- It always defaults to fixing at the deepest level, which is.. not ideal for NPM. On my machine (a not very old Macbook Pro) NPM simply can't update a dependency 20 layers deep in the tree, ie `npm update nested-dependency-from-hell --depth 20` will eventually time out and won't fix anything. So you have to manually crawl up tree yourself and find the thing that can be updated - or just ignore it until the thing right at the top of the tree gets updated.
I'm not surprised to see Dan posting this though. I agree with everything he said, so I don't mean this as an attack, but a lot of the time the thing at the top of the tree we're waiting for an update on is create-react-app. It must be incredibly annoying how many Github issues get opened on that repo every time there's a new NPM advisory on some 20-dependencies-deep parser it uses for something or other.
I do like the suggested fix that a maintainer can use their knowledge of the specific usage to say the vulnerability doesn't apply. Often in these threads there's a perfectly good explanation of why it isn't a real issue, and then people come back with "Okay but can you please update it anyway because I'm forced to audit and my security team/CI are yelling at me".
>> Unfortunately, there are hundreds.
This is primarily a result of the absurd number of dependencies NPM encourages (requires?) people to use. The duplicates are also there in part because of the large number of dependencies and should not be shown more than once by the tool.
Stop building projects with an absurdly large dependency tree, this is just one problem that results from it.
The answer here is probably some kind of static analysis to know which packages end up shipping in the actual bundle to users. I think Dan referenced some work in that regard.
So even more tools?
Javascript seems to prefer millions of tiny dependencies of thousands of larger libraries, which is a choice that can be defended. The difference is not necessarily one in lines of code or binary size, but one in amount of vendors trusted. Many libraries also handle trivial code that (in my opinion) should be part of the programming language or basic developer knowledge already. The is-something packages that fill Javascript dependency trees to the brim can only be considered as failings of the language in my opinion.
As a developer, I trust parties like webpack, gulp, and Facebook, but I haven't heard about jonschlinkert (nothing against him, just a random name I picked) and I don't know who maintains is-number, is-path-cw, is-path-in-cwd, is-path-inside or path-is-inside and how reliable they are. All of these dependencies seem like excellent methods in a library, but they all could've been part of a single dependency no more than 60 lines of code in length. Many NPM packages feel less like libraries and more like automated StackOverflow answers. Adding a vendor to your supply chain for just 40 lines of open source code is just inefficient; why risk trusting yet another vendor to not inject malware in the future like this?
The Java world has some popular names like Apache, Google and Jetbrains that maintain large libraries so it's easy to build a chain of trust. Rust is moving the Javascript way, with hundreds of megabytes of dependencies from thousands of individual repositories, but at least most of its packages add something nontrivial.
C++ doesn't have a package manager, at least not in the same way other languages do. C++ libraries usually come from very specific toolkits or single sources (like Linux package managers). There's tons of packages for C++ development, but all of them are kept up to date by a single organisation on my machine.
Even a simple app with only React as a production requirement will have dozens of issues a month.
There are some packages that don’t have as many dependencies such as Typescript or Prettier, but that’s not enough, since the most popular bundlers have hundreds of of dependencies.
No matter how careful you are, you get flooded by security issues.
If I put a database with default credentials on the internet, there is both a vulnerability and it is exploitable. Bad. If I run a database with default credentials on my dev machine, it is vulnerable, but not exploitable. Perfectly fine.
For real security work you also need to think about impact. Hacker dropping production database = we all lose our jobs. Co-worker connecting to my computer and dropping database as a joke = no real harm done.
So three things to think about: - Vulnerability - Exploitability - Impact
What I really don't like about npm audit is how it presents itself as "security tool" and how vulnerabilities are presented. "6 critical, 10 high vulnerabilities" with a red color screams "fix me now!!!". This is not fair to users because npm has no idea of either the exploitability or the impact of the vulnerability.
Why present users with a prompt "please fix me now!!" and not even mention that exploitability and impact need to be measured first? Seems like they forgot that prompt...
This is like getting mad at the guidebook for showing which plants are weeds when your neighbors complain that your unmaintained garden is full of weeds.
If Dan says "npm audit is a stain on the entire npm ecosystem", maybe it's safe to say that Create React App is a stain on the entire react ecosystem. The best time to maintain it was every single month of its existence because that's how you maintain software.
Facebook has abandoned Create React App. Dan stated that he intentionally does not maintain the project. Rather than complain about npm audit, they should give Create React App over to the community who actually use it instead of keeping it shambling along as a zombie with their name on it.
And if Facebook doesn't want to give it up, the best thing we can do as a community is to move on to any of the other great tools available that are actually maintained.
Wouldn't it be more like getting mad at a guidebook which falsely claims that every plant in your garden is a weed, forcing you to second-guess all of its assertions and therefore wasting a lot of your time?
I've never ever got `audit fix` or `audit fix --force` to solve any of the mentioned vulnerabilities. Ever. I even relied on downloading every dependency one by one to find that there where other offending packages. I just gave up.
It's really useless and deceptive.
That doesn't seem to be the meaning of "broken by design".
There's a push to address the npm audit situation. It's an initiative under the OpenJS Foundation. I kinda started the whole conversation by implementing a tool that makes it acceptable instead of ditching npm audit.
It's called npm-audit-resolver and I've written about it here https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Also check out the collab space and the tool itself https://github.com/openjs-foundation/pkg-vuln-collab-space https://www.npmjs.com/package/npm-audit-resolver
The [1] yarn package manager is much nicer to work with in many more ways.
What are alternatives? A way to ignore or mark a dependency as safe? Could this be abused if an author can just mark a dependency as safe?
Or perhaps, actually analyze syntax with a tool like ESLint (parse -> AST -> validate) to check that dangerous parts of libraries are not in use? This solution comes with it's own complications. Who is authoring these validations?
Perhaps there are other strategies I'm not aware of.
Also going through an audit result in a CLI isn't really the best experience. I wish I could just click a link and open up the report in a browser to drill down into issues.
Don’t like it? Try using more maintainable dependency trees.
In essence, if you are scanning an environment that is already compromised, `npm audit` results can't be relied upon if you are running it in the same environment. It should be self-evident but I'm sure plenty of people use the tool this way.
For a proper assessment, such differences need to be encoded in the security advisory, and the audit tool needs to analyze if the code is called at run time or build time, and then act accordingly.
Because I've felt that way for years.
Well yes, correct, well done. By this metric, every security tool ever written is probably pointless.
I mean, this is why people love language with deep, solid standard libraries. You don't have a situation where a problem in a sub-sub-sub-sub-dependency provokes five different groups of people to all issue an update, one after another. You just upgrade your underlying installation to the latest patch version and continue.
Language ecosystems where a few lines of code constitutes a library fundamentally result in you being dependent on a huge number of outside people to cooperate on updates. That's what's broken. Not a tool which tells you that you have out-of-date libraries and, by the by, hooks into CVE databases.
OP should stop and consider whether it might be beneficial to inline some of those dependencies before dismissing it out-of-hand. If you never use the dependency in such a way as to present a real security risk... and you don't need feature updates from upstream, i.e. the software is fine as-is when you first incorporated the dependency... then why wouldn't you inline the dependency?
If anything, inlining the dependency will allow static code analyzers to point out all the parts of the dependency which you're not using (i.e. dead code) and eliminate it all. That way, even if the dependency were to be discovered to have a security fault, if the faulty code was in a section that you eliminated as dead code... then you don't have a security problem in the first place!
Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.
Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.
There is no way to "mark them as such". That's half of the issue. The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. They just ran `npm install`, and npm adopted default behavior of showing these warnings. For a lot of people this is their first programming environment.
>you can't automate this process, so the only alternative is to ignore all vulnerabilities.
I don't think the answer is necessarily automation. But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
Or at least I'd like npm to offer a reliable way to update packages deep in the tree and override the resolved versions. Currently, there isn't such a way.
This attitude makes me kind of uncomfortable. Like, I have taught software development to a decent number of folks, but I've always done so in a relatively isolated environment. If one is buying into web programming, I have a hard time feeling like it matters that it's their first programming environment--it is a hostile place (the web) and some understanding of that hostility is pretty high on the list, I think, of Things To Get Used To. There's definitely a tension there with "don't overwhelm a novice", but I don't necessarily think optimizing for the novice case is wise, especially when we want those novices to have their heads on a swivel, too.
> But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
I definitely agree with this, though, and this is a good way to help make something like `npm audit` more intelligible and useful.
Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....
Can we please tell beginners not to start programming with node.js?
Teaching beginners to start with “go-to” technologies became industry standard already as it helps corporations to become more and more monopolist and dictate new industry standards.
Or do commenters here actually believe that npm audit should treat a DoS of a development machine as a non-vulnerability?
(Please tell me it’s the former)
I believe that npm audit should treat a DoS of a development machine by a trusted developer as a non-vulnerability. "Code I (or a fellow committer) wrote uses a lot of CPU" isn't a vulnerability. If I care to prevent this, I should run said code within a cgroup with limited resources, not panic about theoretical expense in one part of the codebase while necessarily allowing arbitrary execution elsewhere. "npm audit" is crying wolf, just as the author said.
I like the proposal [1] near the end: "If I own an npm package I need to be able to tag a certain transitive vulnerability category as not affecting my usage of that transitive package." This is particularly important for npm given things like create-react-app but would also be a good idea for "cargo audit" and such.
[1] https://twitter.com/dan_abramov/status/1412380714012594178
Or even, since it's in fact the language itself that sets the tone for the entire ecosystem: javascript: broken by design.
But, one of the goals in software engineering right now is reproducible builds. This means building from source. And of course we'll want to automate that. We've already made inroads with CI
So, correct me if I'm wrong, these are still vulnerabilities.
Tragedy of the commons stuff.
This article might be honest. But I hope in the future we don't need devil's advocate arguments.
All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it.
I think it's totally worthwhile to figure out which tools rely on insecure dependencies.
Also looks like you can specify npm to ignore dev dependencies:
> Any packages in the tree that do not have a version field in their package.json file will be ignored. If any --omit options are specified (either via the --omit config, or one of the shorthands such as --production, --only=dev, and so on), then packages will be omitted from the submitted payload as appropriate.
Quite the opposite! Quoting the article:
As any security professional will tell you, development dependencies actually are an attack vector, and perhaps one of the most dangerous ones because it’s so hard to detect and the code runs with high trust assumptions. This is why the situation is so bad in particular: any real issue gets buried below dozens of non-issues that npm audit is training people and maintainers to ignore. It’s only a matter of time until this happens.
My point is that in the sea of non-issues, real issues are easy to miss and ignore.
>If you don't find this useful that is fine, don't use it.
You can't "not use it" because it's literally the default behavior built into `npm install` now. Of course there are ways to opt out, but this doesn't alleviate the confusion.
It seems like a simple algorithm that works pretty well. Perhaps ignoring certain dependencies makes sense, via an ignore list.
I just find the title "NPM is broken by design" to be a little hyperbolic, when it seems like the complaint is that it's tedious removing all the low-quality dependencies from your project. node security/npm-audit has at least increased the conversation around security for many around the npm ecosystem, where there wasn't much-if-any discussion prior. I think they deserve credit for this.
EDIT: I'm not sure why I'm being downvoted.
A DoS on your build machine and dev machine can be indeed be critical issues. Imagine this scenario:
Your source code is somehow compromised and attackers slip in rogue code to your production site. It siphons off passwords or other PII. The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive, so you can’t just spin a fixed build and re-deploy. You’ll sync your main branch to figure out what is going on, perhaps ready to make a build from your dev machine, but running yarn build hangs. It might take you 1 minute to solve or 5 hours - hard to guess. But every minute you’re delayed is another minute the attacker is siphoning off your production data.
npm audit isn’t perfect, but I don’t agree with the author that devDependencies can’t have critical vulnerabilities. Build machines and dev machines are critical infrastructure. Recall the method of attack of SolarWinds [1].
Related: we all trust that the “many eyes” of open source contributors will keep our dependencies relatively clean, but this function is not infinite. There is some threshold of lines of code and rate of change that will outstrip the community’s natural ability to find and fix problems. I wish the npm community was more sensitive to the risks that are inherent in current practices. Efforts to limit dependencies and perhaps somehow tag which versions have completed a security audit (and by whom) would be great to see.
[1] https://krebsonsecurity.com/2021/01/solarwinds-what-hit-us-c...
Really at this point it's too late to do anything else, instead of trying to dos your dev machine he can instead do simpler things like delete your ssh key from the machine. But let's play along:
> The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive
There is nothing any attacker can do with the static files on the server that will trigger and RegEx DoS in your local development. Aside from the fact that you wouldn't download whatever is on the server back to your machine, even if you did it would never trigger such a DoS since (in the examples in the link) these are modules related to running a dev version of a frontend project based on the raw source files.
Your scenario is only true when an attacker pwned both your production server and your laptop. A regex DoS is really the last thing you worry about at that stage.
> There is nothing any attacker can do with the static files on the server that will trigger a RegEx DoS...
IIUC, an attacker could change my package.json to include inputs to browserlist that trigger a RegEx DoS. To do that, the attacker only needs to make a fraudulent commit. Given how easy most teams make it to commit code, this isn’t too high of a bar.
2. If you are doing a full build and fail because of the regex DOS, then that build would also contain the attacker injected siphoning code, which would make your entire exercise futile in the first place
3. Not obviously messing with the network or crashing build machines would be a better way of siphoning data for longer.
4. This is very contrived.
In the moment, your first thought is that there is some type of quick fix that will restore functionality (if your site is down) or evict the intruder if something funny is detected. As a sibling commenter said, most teams would try to deploy a previous known-good build asset.
But I stand by my point that a DoS of a development system can indeed be critical! I’m surprised to find that I appear to be in the minority here...
