The task benchmarked was from a component of our messaging stack at work that I'd ported. My intention was to offer an example of the end-to-end performance of a real-world task that someone might be writing, rather than a sample task that's not much more than a no-op. I'd suggest that this particular one is real-world as it's straight out of one of our applications. I've no intention of trying to be dishonest here - just offering the measurements of my (and our) internal evaluation of the tool for our needs.
While I can't provide the source of the task, I can offer the quick-and-dirty source of the quick little "PyInvoker" I'd whipped up (sorry - didn't realize there was something called PyInvoke at the time). It just takes a message, unpacks the JSON, and uses getattr to call the appropriate task. Nothing fancy like e-mail error notifications, retries, and the like: https://gist.github.com/18a30689832569d67861
Anyhow, always and absolutely take any claims regarding the performance of a tool with a massive grain of salt, and try them for yourself to see if they suit your needs. Octobot's not designed to replace background processing in most applications as tools like Celery and DelayedJob (both of which I use myself) are great, a bit easier to write for, and a bit simpler to get up and running, depending on the application and language being used.
There's no intention to have slighted anyone or any other project here. I'm stoked that a lot of tools exist in this space. I just hadn't seen one on the JVM that offered this level of simplicity and parallelism with a bit of restraint when it comes to feature creep. But if you have an application that demands high throughput / low latency execution of tasks, this might be worth evaluating.
As to the task selection, I do not think showing a "real-world" task help here, since we all have different tasks to be performed out in the real world. I want to know what overhead a particular piece of infrastructure adds, so something as close as possible to a no-op is a useful data point.
CELERYD_PREFETCH_MULTIPLIER = 0
CELERY_DISABLE_RATE_LIMITS = True
publisher = task.get_publisher()
for i in xrange(1000):
task.apply_async(args=(i, ), publisher=publisher)
publisher.close()
1) State what your project is about using language that a reasonably savvy (technology-wise) user can understand, even if unfamiliar with the problem domain. (this site would be better if they linked on "message queue" for those people that don't know what that is.)
2) Eye-catching call to action icons
3) Brief list of most important features
4) Brief high-level view
5) Brief low-level view
6) Some clean and minimalist stats on why you might want to use it
7) Easy to understand links to more info.
I don't know if I will ever need this product... but I'm bookmarking it anyway as an outstanding example of how to introduce people to it. (Except for the swear word. The word "Fucking" is really out of place)
Nice job guys.
I realize this is not germane to the topic, but .cat!? What other really interesting tld's exist that I have never heard about?
It happens to be not a country code (http://en.wikipedia.org/wiki/List_of_Internet_top-level_doma...), but the generic TLD, alongside the likes of .com and .org, "for Web sites in the Catalan language or related to Catalan culture", the only one of this kind in this category. Go figure.
http://en.wikipedia.org/wiki/.cat - http://en.wikipedia.org/wiki/Catalan_language
I think I need to take a look at the TLDs available, lately, because I know a lot have slipped under my radar.
This project sounds promising for that kind of application but wont know until I try it out.
In our case, we want to be having a very small number of queues that have a high rate of i/o with dozens of reader clients and writer clients.
But guys, seriously, make your docs available in HTML format. PDF-only docs are out of fashion since the 90s.