undefined | Better HN

0 pointsIsaacSchlueter14y ago0 comments

You can open a file with O_EXCL if you pass in the open flags as a number. (You can find them on require("constants"), and they need to be binary-OR'ed together.) This isn't documented. It should be. It should probably also be exposed in a cleaner way.

Most of the rest of what you describe is APIs that need to be polished and refined a bit. The boundaries are well defined at this point, though. We probably won't add another builtin module at this point, or dramatically expand what any of them can do. (I don't consider seek() dramatic, it's just tricky to get right given JavaScript's annoying Number problems.)

0 comments

6 comments · 2 top-level

baudehlo14y ago· 4 in thread

> You can open a file with O_EXCL if you pass in the open flags as a number. (You can find them on require("constants"), and they need to be binary-OR'ed together.) This isn't documented. It should be. It should probably also be exposed in a cleaner way.

That's great to know. Obviously I'm just following the docs.

> Most of the rest of what you describe is APIs that need to be polished and refined a bit

My concern is merely that there have been a number of statements put out saying "we won't be adding anything more to the API", and that we are basically almost at 1.0, at which point there won't ever be anything added to the core API. Lack of flock() is huge (you can't write to an existing file safely without it - and Node developers are doing that all the time, including your own NPM). Lack of an ability to create temporary files safely seems a fundamental weakness - especially when so many Node apps are dealing with file uploads - that's a disaster waiting to happen. We are going to be dealing with Node.js security bugs because of these issues for a VERY long time.

IsaacSchlueterOP14y ago

O_EXCL is in for 0.8: https://github.com/bnoordhuis/node/compare/O_EXCL

flock() is not trivial to do in a portable way. For a unix-only flock(), check out the fs-ext addon. Same for mktemp.

I wouldn't be opposed to either being in core if it could be done in a clean way, but this is just adding another knob that can be done with an addon easily enough. If you care more about having flock() than about writing portable programs, then that's what the fs-ext addon is for.

> We are going to be dealing with Node.js security bugs because of these issues for a VERY long time.

Of course we'll be "dealing with Node.js security bugs for a VERY long time", because we'll be using Node.js for a very long time. Software is buggy, and many bugs are security hazards. We'll be dealing with "Unix security bugs" and "C security bugs" and "Java security bugs" forever as well.

Please do not make vague suggestions about security issues. Either you've found an issue, and should be submitting it, or you haven't, and are just spreading fud.

baudehlo14y ago

> flock() is not trivial to do in a portable way

Perl, Python and Ruby manage it.

> For a unix-only flock(), check out the fs-ext addon

Which I wrote.

> Of course we'll be "dealing with Node.js security bugs for a VERY long time", because we'll be using Node.js for a very long time.

That's not quite what I meant - I mean that people right now are writing temp files in LOTS of Node.js applications in an insecure way. It's good that O_EXCL is available, I'll try and submit a patch to node-temp, but really temp file creation should be in core (amongst other things).

This isn't a vague suggestion. There are COUNTLESS security bugs created every day by insecure temp file creation. Let's see, from npm these packages have security bugs because they rely on the insecure node-temp: ShipItJS assetgraph-builder confy filerepl gracie joose js-loader muffin nerve redisfs.

bcantrill14y ago

I disagree that it is "huge" that flock() is missing; if you are relying on advisory (emphasis on advisory) file and record locking to implement coherence, then your architecture is likely an accident waiting to happen -- it's a highly problematic interface for many reasons. Of course, if you really need it, use fs-ext as Isaac mentioned -- NS more generally, if you need some system functionality that isn't present in core, develop an add-on and knock yourself out. It's not entirely pleasant, of course -- but don't pretend that an interface not being in core means that some bit of system functionality is somehow off limits...

jclulow14y ago

Huge might be the wrong word, but I've definitely been frustrated that it's missing. Java has a standard library class that works well enough for whole-file advisory (or mandatory if you're on Windows, I suppose) locks on a local filesystem. It's eminently helpful for excluding a second process from starting with a lock that the OS will clean up for you if/when your process terminates for whatever reason.

smf(5) can get you part of the way there, but not if you're (say) writing some system bits that you need to deploy on platforms in addition to SunOS. Plus, if you can avoid depending on any not-just-pure-JS modules then you can deploy one tree onto all of your platforms without additional build steps, C++ compile/ABI issues, etc.

craigyk14y ago

THANK YOU. This addresses a question I've had on stack overflow for a couple of days. It also addresses at least two issues in Node.js's github tracker under the 'fs' flag.

j / k navigate · click thread line to collapse