undefined | Better HN

0 pointsMrJohz1y ago0 comments

It's worth noting that Node allows two packages to have the same dependency at different versions, which means that `npm i -g` is typically a lot safer than a global `pip install`, because each package will essentially create its own dependency tree, isolated from other packages. In practice, NPM has a deduplication process that makes this more complicated, and so you can run into issues (although I believe other package managers can handle this better), but I rarely run into issues with this.

That said, I agree that `npm i -g` is a poor system package manager, and you should typically be using Homebrew or whatever package manager makes the most sense on your system. That said, `npx` is a good alternative if you just want to run a command quickly to try it out or something like that.

0 comments

zahlman1y ago

>It's worth noting that Node allows two packages to have the same dependency at different versions

Yes. It does this because JavaScript enables it - the default import syntax uses a file path.

Python's default import syntax uses symbolic names. That allows you to do fun things like split a package across the filesystem, import from a zip file, and write custom importers, but doesn't offer a clean way to specify the version you want to import. So Pip doesn't try to install multiple versions either (which saves it the hassle of trying to namespace them). You could set up a system to make it work, but it'd be difficult and incredibly ugly.

Some other language ecosystems don't have this problem because the import is resolved at compile time instead.

MrJohzOP1y ago

This is incorrect on several points.

Firstly, Node's `require` syntax predates the modern JS `import` syntax. It is related to some attempts at the time to create tools that could act like a module system for the browser (in particular RequireJS), but it is distinct in that a lot of the rules about how modules would be resolved were specifically designed to make sense with NodeJS and the `node_modules` system.

Secondly, although path imports are used for local imports, Node's `require` and `import` syntax both use symbolic names to refer to third-party packages (as well as built-in packages). If you have a module called `lodash`, you would write something like `import "lodash"`, and Node will resolve the name "lodash" to the correct location. This behaviour is in principle the same as Python's — the only change is the resolution logic, which is Node-specific, and not set by the Javascript language at all.

The part of NodeJS that _does_ enable this behaviour is the scoped module installation. Conceptually, NPM install modules in this structure:

    * index.js (your code goes here)
    * node_modules (top level third-party modules go here)
        * react
            * index.js (react source code)
            * node_modules (react's dependencies go here)
                * lodash/index.js
        * vuejs
            * index.js (vuejs source code)
            * node_modules (vue's dependencies go here)
                * lodash/index.js
        * lodash
            * index.js

Importantly, you can see that each dependency gets their own set of dependencies. When `node_modules/react/index.js` imports lodash, that will resolve to the copy of lodash installed within the react folder in the dependency tree. That way, React, VueJS, and the top-level package can all have their own, different versions of lodash.

Of course in practice, you usually don't want lots of versions of the same library floating around, so NPM also has a deduping system that attempts to combine compatible modules (e.g. if the version ranges for React, Vue, and Lodash overlap, a version will be chosen that works for all modules). In addition, there are various ways of removing excess copies of modules — by default, NPM flattens the tree in a way that allows multiple modules to "see" the same imported module, and tools like PNPM use symlinks to remove duplicated modules. And you mention custom importers in Python, but I believe Yarn uses them already in NodeJS to do the fun things you talk about like importing from zip files that are vendored in a repository.

All of the above would be possible in Python as well, without changing the syntax or the semantics of the `import` statement at all. However, it would probably break a lot of the ecosystem assumptions about where modules live and how they're packaged. And it's not necessarily required to fix the core Python packaging issues, so I don't think people in Python-land are investing much time in exploring this issue.

Basically, no, there is no fundamental reason why Python and Node have to have fundamentally different import systems. The two languages process imports in a very similar way, with the main difference being whether packages are imported in a nested way (Node) or as a flattened directory (Python).

j / k navigate · click thread line to collapse