Key/value is dead. Long live tuples: Pangool for Hadoop (opens in new tab)

Dead is relative. Dead usually means "dead to me".

So, what you're saying is "'X is dead' is considered harmful"?

And for a slightly more modern take on the tuple space, check out Java Spaces [1] or Gigaspaces [2]. There's still plenty of active research on the topic too [3] (disclaimer: I did my PhD thesis on distributed tuple spaces).

I've long contended that a tuple space was basically a generalised key-value store, so it's nice to see projects like this one crop up.

[1] http://java.net/projects/jini/

[2] http://www.gigaspaces.com/

[3] http://eprints.utas.edu.au/9996/

Apache Pig is a much different beast than this project, from what I can tell reading the documentation for Pangool. While they both operate on tuples and work at a higher level than pure Hadoop, they accomplish their goals much differently.Pig uses its own language called Pig Latin (http://pig.apache.org/docs/r0.9.2/basic.html), which is then compiled down into code that interfaces with the Hadoop library. Pangool is much closer to Hadoop, in that you are writing Java. If you look at one of their examples (http://pangool.net/introduction.html), I get the sense that the developers aim to make Hadoop easier to use, while Pig aims to make data analysis easier to use.

These goals are greatly divergent. In Pig, Java code is written to create new functions that can be used for analysis--i.e. Java is written in support of Pig Latin. Pangool focuses instead on extending Hadoop by making the Java code easier to write. This means Pig could potentially be implemented in Pangool, if Pangool were to satisfy the requirements for the task. (Not that I am suggesting that Pig actually be written--it might just be possible, depending on the technical requirements.)

Having used Hadoop in the past, I would be more inclined to use Pangool. Parts of Hadoop are poorly written--especially the reliance on singletons--and anything that makes it easier to write code that runs on a Hadoop cluster is a desirable goal in my eyes. I look forward to seeing how this project shapes up.

Hi, I'm one of the developers of Pangool. The idea of Pangool is not to be yet another higher level API on top of Hadoop but rather to pose a replacement for the low-level Hadoop Java MapReduce API. Pangool has the same performance and flexibility than that of the Java MapReduce API although it makes several things a lot easier and convenient. There is no tradeoff, just advantages. There will be cases where you'd want to use Pig or Cascading. There will be some other cases where you'd want the flexibility and efficiency of MapReduce. For those cases we conceived Pangool. Nowadays only very advanced Hadoop users could write efficiently-performing MapReduce Jobs. Pangool hides all the advanced boilerplate code needed for writing highly efficient MapReduce jobs, making things like secondary sorting or reduce-side joins extremely easy.

Hi rjurney. I would say "Hive, Pig, Cascading" are on the higher level API side and "Pangool, MapReduce" on the low-level side. Pangool is a MapReduce API that aims to make MapReduce simpler. We explain this better in our FAQ: http://pangool.net/faq.html

Exactly. A tuple is exactly the same as a relation.