We actually extracted our promises implementation from the work we've been doing, and released it as RSVP.js[1]. While other JavaScript promises libraries are great, we specifically designed RSVP.js to be a lightweight primitive that can be embedded and used by other libraries. Effectively, it implements only what's needed to pass the Promises/A+ spec[2]. For a comparison of RSVP.js with other promises-based JavaScript asynchrony libraries, see this previous discussion on Hacker News[3].
1: https://github.com/tildeio/rsvp.js
If I wanted to compute the size of one file relative to a set, I’d probably do something like this:
queue()
.defer(fs.stat, "file1.txt")
.defer(fs.stat, "file2.txt")
.defer(fs.stat, "file3.txt")
.awaitAll(function(error, stats) {
if (error) throw error;
console.log(stats[0].size / stats.reduce(function(p, v) { return p + v.size; }, 0));
});
var q = queue();
files.forEach(function(f) { q.defer(fs.stat, f); });
q.awaitAll(…); // as before
A related question is whether you actually want to parallelize access to the file system. Stat'ing might be okay, but reading files in parallel would presumably be slower since you'd be jumping around on disk. (Although, with SSDs, YMMV.) A nice aspect of queue-async is that you can specify the parallelism in the queue constructor, so if you only want one task at a time, it’s as simple as queue(1) rather than queue(). This is not a data dependency, but an optimization based on the characteristics of the underlying system.
Anyway, I actually like promises in theory. I just feel like they might be a bit heavy-weight and a lot of API surface area to solve this particular problem. (For that matter, I created queue-async because I wanted something even more minimal than Caolan’s async, and to avoid code transpilation as with Tame.) Callbacks are surely the minimalist solution for serialized asynchronous tasks, and for managing parallelization, I like being able to exercise my preference.
There is a lot of engineering that goes into making parallel reads go fast. Some combination of the file system and disk controller will probably be smart enough to recognize the opportunity for sequential reads and execute them as such if possible.
This is not always true, and it does not undermine the rest of what you have written. I just think it's interesting to keep in mind that operating systems implement a lot of helpful machinery that user-level programmers forgot about.
do f1 <- fsStat "file1.txt"
f2 <- fsStat "file2.txt"
f3 <- fsStat "file3.txt"
let ratio = (size f1) / (sum $ map size [f1, f2, f3])
print ratio
do fs <- mapM fsStat files
let ratio = (size . head $ fs) / (sum . map size $ fs)
print ratio
Perhaps people don't spot the link as easily because monads are usually explained in terms of a type system, and javascript is untyped? (Or perhaps just because Monad is a very abstract abstraction :)
The surface area is `.then()`
var queue = new Promises;
fs.stat("file1.txt", queue.cb());
fs.stat("file2.txt", queue.cb());
fs.stat("file2.txt", queue.cb());
queue.all()
.then()
.fail();
pub.on('foo', function() {
promise1.fulfill();
});
pub.on('bar', function() {
promise2.fullfill();
});
Vow.all([promise1, promise2]).then(...).fail(...); > If foo takes many arguments we add more arrows, i.e. foo :: a -> b -> c
> means that foo takes two arguments of types a and b and returns something of
> type c.
Other than that teeny thing, this article is awesome, and I fully agree. Promises are an excellent thing, and while I'm just getting going with large amounts of JavaScript, they seem far superior to me.
Whilst that's true, and is important to the way in which Haskell operates, people normally talk about functions as taking multiple arguments (at least, the people at London HUG, most of whom are better Haskellers than I, seem to).
Even ghci refers to the "second argument":
Couldn't match expected type `Int' with actual type `Char'
In the second argument of `foo', namely 'b'
In the expression: foo 1 'b'
In an equation for `it': it = foo 1 'b'Often, conversations are not held to absolute rigor. Not every off-handed statement is absolutely consistent.
f :: a -> b -> a
f a = g a
g :: a -> b -> a
g a _ = a
If you have some crazy graph of dependencies, I can see how breaking out promises could help simplify things. But I don't feel like that's a super-common scenario.
The author says:
> * [Promises] are easier to think about precisely because we’ve delegated part of our thought process to the machine. When using the async module, our thought process is:*
> A. The tasks in this program depend on each other like so,
> B. Therefore the operations must be ordered like so,
> C. Therefore let’s write code to express B.
> Using graphs of dependent promises lets you skip step B altogether.
But in most cases, I don't want to skip B. As a programmer, I generally find myself preferring to know what order things are happening in. At most, I'll parallelize a few of database calls or RPC's, but it's never that complex. (And normal async-helper libraries work just fine.)
I swear I want to wrap my head around how this promises stuff could be useful in everyday, "normal" webserver programming, but it just always feels like over-abstraction to me, obfuscating what the code is actually doing, hindering more than helping. I want to know, specifically, if one query is running before another, or after another, or in parallel -- web programming is almost entirely about side effects, at least in my experience, so these things often matter an awful lot.
I'm still waiting for a real-world example of where promises help with the kind of everyday webserver (or client) programming which the vast majority of programmers actually do.
> Getting the result out of a callback- or event-based function basically means “being in the right place at the right time”. If you bind your event listener after the result event has been fired, or you don’t have code in the right place in a callback, then tough luck, you missed the result. This sort of thing plagues people writing HTTP servers in Node. If you don’t get your control flow right, your program breaks.
I have literally never had this problem. I don't think it really plagues people writing HTTP servers. I mean, you really don't know what you're doing if you try to bind your event listener after a callback has fired. Remember, callbacks only ever fire AFTER your current imperative code has finished executing, and you've returned control to node.js.
Promises make async code easy to manage, even at scale. Each API request gets its own promise. What happens inside that promise doesn't matter, as long as it returns a result or an error. If talking to the API takes one request or two, it does not matter with promises. We can abstract these API requests in such a way that even if document retrieval is a multi-step process or a one-step process for each document source, the collation process can know nothing about the retrieval process to work.
In other words, promises allow us to separate concerns. Document retrieval is one concern, collation another. Promises, by abstracting asynchronous control flow into synchronously appearing objects, allow us to write simple programs at higher levels. Separating concerns makes the server easier to test and easier to extend.
If you want to see how I've used promises, you can take a look at my work on Github [2].
Other programming languages have this too. They're called a 'METHOD'. Sorry, couldn't resist.
On a serious note, look at your code in here:
https://github.com/fruchtose/muxamp/blob/master/lib/playlist...
And look at your 'playlistCount' function on line 39 (which for no apparent reason you've made a variable).
Now I can't understand how a promise there is helping your programming. In fact it seems to have vastly increased the complexity of the code and it is now completely unclear from a quick scan how the method works, what the program flow actually is.
It should be a much shorter function, literally just this:
function playlistCount(){
return db.query('SELECT COUNT(id) AS count FROM Playlists;', function(rows) {
return parseInt(rows[0]['count']);
});
}
28 LINES. HOW? For a simple COUNT!
Actually to be fair to promises, 50% of the problem here is that you've not abstracted your db code and unnecessarily repeat boiler plate connection code again and again and again.
But a big part of the problem is that you can't just throw an exception and let it bubble up the stack, you're constantly having to baby sit the code, and that is the promises fault.
(unrelated, but as a mini-code review though I've never used node.js but the oft repeated line of `dbConnectionPool.release(connection);` in the connection failure code is an immediate code smell to me, it looks like rain dance code, why would you have to release a failed connection? It failed, how can it be released?)
The idea that webservers are "all about side effects" gives me a chill. The whole architecture concept of HTTP is no side effects, so to claim that it's all about side effects seems odd. It should only be the case for POST PUT or DELETE methods, and only in very specific ways.
However, when you learn to stop worrying and let the runtime decide it's so much nicer. It turns out that people have already optimised the framework, so at worst it's just as fast as the code I wrote. At best it's faster because the framework knows more about what it's capable of.
The biggest thing to realise is that while you can easily make things perform well in isolation the runtime can look at the bigger picture. There's no point making an operation run in 300ms if it blocks all other tasks on the server, when it could run in 600ms and allow everything else to keep going.
This smells like the classic leaky abstraction though. Like when people tried to paper over the difference between remote calls and local calls with abstract interfaces (CORBA, RMI, etc.). Everyone would say it was so awesome, remote calls look the same as local calls! But it wasn't awesome, it was horrible, because the details of the abstraction 'leaked' through and you got all kinds of problems from having delegated away what was actually one of the most critical, sensitive parts of your code to a layer you had no control over. 15 years later we're back to nearly everybody using REST because it turns out to be way better not to shove those abstractions on top of your most important code.
Now, I'm not saying that analogy is perfect here ... but it does remind me of it. Why should you care about the order of things? Just to suggest something, sometimes it's just useful to be able to reason about it. "We know the first operation definitely happened before the others, so an earlier one failing can't be a side effect of something a later one did ... oops, we don't know that any more. We actually have no idea what order they happened in."
There's nothing incongruous about that. It is the case that side effects should only happen on POST, PUT, and DELETE methods (and the like), but almost all webservers are written because of a need to use these.
If your webserver is all GETs and HEADs, then it is either trivial and you would have used someone else's instead of writing your own, or its sole purpose is to repackage and serve existing data from other sources - a rare use case among all webservers.
If you were to take an inventory of all the webservers out there, you would doubtless find that almost all of them exist in large part in order to create side effects.
Using a Promises, as opposed to reducing a list of computations async-style, also limits you to the Promise object's interface, so you lose (or at least add cruft to) the flexibility and composability of using native lists. By sequencing computations with lists, if I want some insight into what's happening, I just List.map(apply compose, logFunc). With promises, I have some work to do.
Promises have their uses, but it's definitely a tradeoff, and for most HTTP servers, I'd argue that their utility does seem a bit limited. I'd similarly say that making a point of using FRP to build a server would probably be a bit overkill for the task.
One answer would be to just do them synchronously, perhaps nesting one of the api calls inside the other's callback (event-handler or otherwise). And then for the Default view, you'd need to have the code that handles the failure in 2 different places (or at least 2 tests for failure).
Another answer would be a state machine which can be grouped and chained (pipe'd) with other similar state machines, to guarantee an order of operations when you need it. With this, you create a promise which is only resolved when both of the promises for the 2 ajax calls are complete, and that promise then pipes the results to the next promise in the chain which renders your view. For the Default view, if either of the ajax calls fail then the parent promise will fail, allowing you to handle the failure in one place.
Code example:
Deferred.when( ajax1(), ajax2() ).then( /* success */ mainView(), /* fail */ defaultView() );Anytime you need to perform more than one interaction with external services in parallel, it's a lot easier to wrap the results as futures and interact with the promise objects than to cope with callback spaghetti.
This is particularly critical anytime you're working in an SOA environment.
I could see why it's easy to believe that you don't need promises if you're averaging 1-2 database queries and 0 API calls per web page/API call reply, but once your scenario gets even slightly more complex - you're fucked.
Promises are one of the big reasons I like Clojure's concurrency better than Go's.
Alan Kay (yes, that Alan Kay[1] -- the guy that's won a Turing award) formalized spreadsheets as a limited form of first-order functional programming.[0]
Kay's interest in spreadsheets wasn't about functional programming, it was about interactive and dynamic computation. I have a pdf of the 1984 Scientific American article that Wikipedia is quoting from. It does include the phrase "value rule"—by which he simply meant what we would call a spreadsheet formula—but I'm pretty sure it makes no argument about functional programming (it's all images so I can't search to be sure). If you'd like a copy, email me. It's a pretty neat article, ahead of its time as one would expect from Alan Kay.
In that case any languages supporting function is "first-order functional programming." It's kind of meaningless when we are talking about real functional languages. PHP can define a function that takes a value and returns a value, and thus it can do "first-order functional programming." But no one claims PHP is a functional language. (Not to degrade PHP, it's a good language doing what it does best.)
All these twisting the definition of functional programming just produce more confusion.
Whereas in functional languages, the functions declare relationships between values and the language uses the evaluation model to propagate the results between function evaluation. Where's the difference?
A mathematical (ie pure) function is declarative in nature as it defines a relation between sets. However, in practice you often use it imperatively, ie as a means to get an output from some input.
An object is imperative in nature as it encapsulates mutable state. However, the set of messages defines an abstract interface which is arguably declarative.
In principle, a purely functional language cannot be object-oriented, and a purely object-oriented language cannot be functional. In practice, this doesn't matter as pure languages are rare.
It's not functional at all. This reaffirms my belief that blog posts are a terrible place to learn. People who know the least shout the loudest.
Functional programming is declarative. Especially when it's lazy and the program's instantaneous state is abstracted out. One of the motivating goals in functional programming is to be able to define a computation once, in terms of other computations, and have that relationship be maintained with minimal regard to the state of the program or its order of execution. Which is what it seems like (this is the first time I've specifically encountered them) Promises are a powerful tool for accomplishing.
In contrast, threading explicit callbacks/continuations through a program, which is explicitly managing the order of execution, is relatively more imperative, which I think is the point of the article -- you don't need the wider control-flow flexibility of explicitly and manually threading callbacks to do the type of computations that most async web stuff does. You can abstract the common callback pattern out into something like Promises, and make all your shit more consistent and concise.
Because many people (myself included) only want a low-level interface
to file system operations that does not necessitate creating an
object, while many other people want something like promises but
different in one way or another. So instead of promises we'll use last
argument callbacks and consign the task of building better abstraction
layers to user libraries.
Yes, there is: https://github.com/promises-aplus/promises-spec
Promises don't somehow magically make asynchronous code easy to write while leaving callbacks out in the cold. They have very similar strengths and weaknesses and I didn't find any of the OP's arguments compelling.
In fact, if I had to choose, I would take the opposite view and say callbacks are neater, cleaner and more consistent than promises.
I actually didn't know about futures until I learned about scala and finagle. I watched a talk on twitter's service stack given by marius eriksen and was blown away. My coworkers heard me rambling on about futures for weeks afterwards, and I found that it was difficult to explain what was so great about them. So I'm not surprised at the negative reactions in the comments here (although jcoglan did a much better job of exlaining them than I ever did).
// list :: [Promise a] -> Promise [a]
var list = function(promises) {
var listPromise = new Promise();
for (var k in listPromise) promises[k] = listPromise[k];
for (var k in promises) listPromise[k] = promises[k];Promise libraries, like RSVP.js [1] he referred to, typically implement a way to construct a promise with a depends-on-many relationship, as a function possibly called `all([p1, p2, ...])` (with the same type signature as for `list`), `and(p1, p2, ...)` or something similar.
IMO, defining the `list` function that way would've been clearer to the reader and more FP'ish, treating the `promises` argument in as a value and not a mutable object.
[1]: https://github.com/tildeio/rsvp.js/blob/master/lib/rsvp/all....
var listPromise = new Promise();
for (var k in listPromise) promises[k] = listPromise[k];
Coming from a Python background, I think I would have found this more obvious if the variable "k" were instead called "method" or "attr". If it was `for (var method in listPromise)` it'd be much clearer what's going on, whereas single-letter variables like i, j, and k are, to me, stand-ins for integers.
It was also confusing, as you said, that the function uses destructive update rather than treating the input as a value. James did mention this ("augmenting the list with promise methods"), but it's still unexpected, especially when the function is preceded by a Haskell type signature.
The reason I only say I'm closer to understanding, and not quite there yet, is I'm not sure what it means to do `new Promise()` or what is being copied over in the above for-loop. I tried James's code with a Promises/A+ implementation, rsvp.js (https://github.com/tildeio/rsvp.js), but it won't let me do `new Promise()` because it works differently:
> var promise = new RSVP.Promise();
TypeError: You must pass a resolver function as the sole argument to the promise constructor
var promise = new RSVP.Promise(function(resolve, reject){
// set up a callback that calls either resolve(...)
// or reject(...)
});
I still think this is a great article, but that code snippet has proven to be quite a puzzle.
This is especially noticeable when you have branching behavior / want to resolve a promise early[1]:
Branching with promises:
function doTask(task, callback) {
return Q.ncall(task.step1, task)
.then(function(result1) {
if (result1) {
return result1;
} else {
return continueTasks(task);
}
})
.nodeify(callback)
}
function continueTasks(task) {
return Q.ncall(task.step2, task);
.then(function(result2) {
return Q.ncall(task.step3, task);
})
}
function doTask(task, callback) {
$$([
$$.stepCall(task.step1),
function($, result1) {
if (result1) return $.end(null, result1)
},
$$.stepCall(task.step2),
$$.stepCall(task.step3)
], callback)
}
[1] http://stackoverflow.com/questions/11302271/how-to-properly-...
[2] https://github.com/Schoonology/stepdown (docs need updating, view tests for documentation)
SomeClass.prototype.someActionPromise = function(){
var deferred = makeADeferred();
SomeClass.prototype.someAction.call(this, function(err){
err ? deferred.reject() : deferred.resolve();
});
return deferred.promise();
};
I'm interested in your opinion RE: his argument for why callback APIs are imperative - because I think he has a very good point and has supported it with a solid argument and you haven't offered any rebuttal.
After reading it, I think I have to agree that I had never thought about it that way. It makes a lot more sense that a promise is just a declaration of some unit of work, and when you can use a promise like any other data, you aren't just giving imperative commands, but rather describing work to be done and using that as a fundamental part of your code, which is why it drastically simplifies async programming (the relation of promises to monads is quite nice, too). Definitely a good article, thanks for kicking me :).
Rewind to the point when nodejs was being designed. In that world, in the context of javascript, callbacks were the only real pattern that existed. XHR? callback. Doing something in the future? callback.
If you imagine node trying to leverage the javascript ecosystem, callbacks were a no-brainer.
The XHR object is effectively a request and a response bundled up into one object that has promise-like traits. You attach event handlers to it to handle various state changes and scenarios, and then once you issue the request, the event handlers get invoked 0-N times. If it really was JavaScript callback-style, XHR would look like this:
window.xmlHttpRequest("GET", url, function (result, error) { ... } );
Honestly, the only way to classify node's callback-heavy design as a 'no-brainer' is if you excuse the design by saying no thought was put into it beyond simply doing what a bunch of other people were doing. If you put enough thought into how large applications will be built, and how difficult it is to build scalable, maintainable libraries, callback-passing style easily loses compared to promises.
Here's a discussion about removing promises from node.js in 2010:
https://groups.google.com/d/msg/nodejs/RvNoQtoWyZA/ar_lYLhK8...
Sure, but there were languages back then that made programming with callbacks more pleasant due to features like coroutines or first class continuations. While these might not have been on the radar for most Javascript developers the technology certainly existed back then.
See the second paragraph of the article for the authors take on this. Specifically, the concept of values is very important.
It's one of the reasons why i started learning haskell.
var result = [];
paths.forEach(function (i, file){
fs.stat(file, function (err, data){
result.push(data);
if (i === 0) {
// Use stat size
}
if (result.length === paths.length) {
// Use the stats
}
});
});
var result = [];
var hasFailed = false;
paths.forEach(function (i, file){
fs.stat(file, function (err, data){
// if previous callback failed, give up.
// unless this is the first item, which we still need
if(hasFailed && i !== 0) return;
if(err) {
hasFailed = true;
// do something with the error
if (i === 0) // do something special for error on first item.
return;
}
result.push(data);
if (i === 0) {
// Use stat size
// remember we might have already failed, in which case don't add the first item to the general result
if(hasFailed) return;
}
if (result.length === paths.length) {
// Use the stats
}
});
});
And your example is disingenuous too, why do we need to add so much logic instead of collecting the errors if you want to handle them anyway?
result.push(err || data);
var failed = results.every(Buffer.isBuffer);Consider your use of the forEach() abstraction when you could have used a for() loop just as easily. (That said, I agree the article in general could do a better job of describing the "other" side)
And in such case you would only do this once outside the listeners of http/or-whatever connections so it would be done just once regardless of the number of concurrent activity.
Why don't we all just go back to manually pushing/popping arguments on the stack like in assembly language while we're at it?