Stencila – Spreadsheet-like live reactive programming environment (opens in new tab)

Lotus 1-2-3 got people to use computers. One could argue that spreadsheets are the single biggest business innovation of the late 20th century. I don't think they're going away that easily.

I know people who type whole letters in Excel. I know an accountant in particular who I showed how to use MS Word for writing letters, but he prefers Excel. He writes full documents in Excel, prints them out and mails them.

Laugh if you want but spreadsheets are not going away in my lifetime.

Spreadsheets are optimized for data entry. Try entering CVS data manually using a text editor.

Without Excel there would be no way for my sales/marketing team to give me valuable data to process at the backend.

If you have been doing programming in excel for the past 20 years then you are simply a fool - sorry.

I think the more usual form of that phrase is "Spreadsheets are dead, long live spreadsheets!". And that captures the truth of the situation better.

Despite all the known weaknesses of spreadsheets, it is sheer hubris to believe that they will be supplanted by reactive programming (or databases, or anything else for that matter). It seems that users will give up their spreadsheets only when they are pried from their cold, dead hands.

European Spreadsheet Risks Interest Group has some great links to articles about the pros/cons/whattodo of spreadsheets: http://www.eusprig.org/

Jocelyn Ireson-Paine has done lots of work with spreadsheets and their problems. Lots of links to spreadsheet sites from there: http://www.j-paine.org/

Iteration on spreadsheets is great, and there's opportunity for tons of innovation in this space. But this suffers from the same problems that most spreadsheets have. Errors like not averaging over the right set of data (which you pointed out) don't come up because the code can't be diffed in git. They come up because there's not enough clarity in whether a set of cells are consistent or not. Making this text-based doesn't add clarity because ultimately there will be too much text to read.

Here's a test for the most common kind of error. If I add data in A5 and B5, will that data be represented in the chart? How about in the averages? Will I be able to even see that?

Here's the pivot: Break your data into regions, where regions have repeated elements. Something like this:

    A1 = 'Height
    B1 = 'Width
    C1 = 'Area
    A2:C2 = Region {
      C1 = =A1*B1
    }{
      A1 = 2
      B1 = 3
      A2 = 4
      B2 = 5
    }
    B3 = 'Average
    C3 = =sum(C2)/count(C2)

This could be used to generate:

    Height  Width   Area
    2       3        6
    4       5       20
            Average 13

The dataset associated with that sub-block can be clearly annotated to an editing user as such, with simple tools for adding a row to that dataset, or sorting it without effecting the rest of the sheet. Within the dataset, there's no corruption of the formulas (the third row's C cant be different than the second's), you've still got your diffing (probably better because it's clear that data changed but formulas different) and it's extremely hard to make the calculated average come out wrong.

Yeah, that exact representation isn't great. Maybe a "view" metaphor, so that headers and footers can be attached to the dataset instead of floating outside it. But once you've gone this direction, there's all sorts of amazing things possible by sharing datasets, linking them, transforming them, etc.

I think another change is needed for spreadsheets to be a more reliable tool.

Right now a sheet is a full document thing. Meaning that it starts with A1 in top left, and spreads out from there.

but what if you could break a "sheet" down into units?

So that you can have say a constants unit, with its own A1 to whatever as needed, and then another unit, perhaps called results, that hold the formula in their own A1 to whatever, each unit movable and extendable on the screen at the same time.

Now if formula A1 wants to refer to something in constants A1, the entry would look something like constants(A1).

This way one would have everything on screen, while avoiding the worry that as the sheet grows and one move things around to make it more readable, the formulas are referencing the wrong cells.

OT and just my opinion: it seems to exists an unfortunate trend in the promotion of new ideas where, instead of simply exposing the virtues of the idea on its own, it first needs to knock down the (proven and world adopted) predecessor as something unpolished, unplanned, untested and, in general, a bad idea from the start.

Compare this trend, for instance, with the first Linux announcement by Linus Torvalds [1] below.

It is an unfortunate trend because it ends up polarizing and creating unnecessary division among the early adopters of the new idea and the current userbase of the old one.

From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds)

Newsgroups: comp.os.minix

Subject: What would you like to see most in minix?

Summary: small poll for my new operating system

Message-ID: <1991Aug25.205708.9541@klaava.Helsinki.FI>

Date: 25 Aug 91 20:57:08 GMT

Organization: University of Helsinki

Hello everybody out there using minix –

I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I’d like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

I’ve currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I’ll get something practical within a few months, and I’d like to know what features most people would want. Any suggestions are welcome, but I won’t promise I’ll implement them :-)

Linus (torvalds@kruuna.helsinki.fi)

PS. Yes – it’s free of any minix code, and it has a multi-threaded fs. It is NOT protable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that’s all I have :-(.

[1] http://www.thelinuxdaily.com/2010/04/the-first-linux-announc...

Hey, Stencila developer here. Thanks to the original poster for sharing the link and for all the interest. Unfortunately, the site is not handling all that interest too well (R session hosting instances filling up, timeouts,...). I'm working on it but please bear with me while I try to stabilize things.

I've been shifting a bunch of data analysis out of spreadsheets (and some adhoc SQL) to Jupyter/Pandas, and we've found some unexpected tradeoffs on both sides.

The lack of testability and version control (and really, long-term maintainability) is what drove us out of spreadsheets into Jupyter. We've found though that the workflow in Jupyter, even with Pandas, is not great for exploring and getting a feel for data --- we end up missing the very quick "poking around" you can do in excel or in a good sql client.

I'd have to use Stencila more to know if it strikes the right balance for the kind of analysis work I do, but I'm glad to see such a thoughtful attempt to try a new balance.

This article reminded me of an interesting video by Chris Granger, co-author of the Light Table code editor, titled "In Search of Tomorrow: What does programming look like in 10 years?" He's designing a visual/code block hybrid editor.

https://www.youtube.com/watch?v=VZQoAKJPbh8

I am personally a fan of Calca, not only does it do live coding, it can also do some powerful math:

http://calca.io/

This looks very sexy, and I don't say that often about technology. This is something I've wanted to see for a long time - the ability to seamlessly move from a spreadsheet to an application

Edit: Quick feedback

Cell names appear to be case sensitive. While this makes sense from a programming standpoint, if I can't have a cell named "a1" the application should convert that to "A1" for me.

Not being able to tab between cells is annoying. Was there a decision made not to capture the tab event?

This looks very similar to AlphaSheets and the thing I was making over spring break until my friends called it stupid.

Very interesting!

Off-topic: how do people make those animated gifs like the one in the blog post (https://stenci.la/stencila/blog/introducing-sheets/screencas...)?

hi - quick question. Is your Dockerfile open source ? We have been building an R powered backend for one of our analytics services and it's been a struggle getting a scalable R-based infrastructure going.

BTW - another killer usecase would be an easy way to build database backed spreadsheets. instead of the TSV, it would be cool to figure out how to interface to postgresql for example.

I think notebook environments like iPython provide greater flexibility in terms of programming. I don't think they can replace spreadsheet nor spreadsheet would develop functionality to replace notebook environments.

The missing link is a seamless experience between spreadsheet and reproducible data analysis (programming).

My team is working on https://nxsheet.com. We look to provide this seamless experience. For example: Generate normal distribution - https://nxsheet.com/sheets/56e845da4030182e337c6c2b

Stencila looks interesting. Great work!

Has anyone here ever used Lotus Improv? Does this program solve some of the flaws that this article highlights? I keep on hearing how Improv was one of the great programs that never took off, and I'm curious to now try it.

For those of you that don't know (or weren't even born yet), Lotus Improv was a spreadsheet alternative that was released for the NeXT computer in the early '90s that was well-reviewed but never sold well. It was eventually abandoned when IBM bought lotus in the mid-90's.

It seems, perhaps, Resolver One's time has finally come. Alas, Resolver One (and Resolver Systems) died a slow death and ceased to be in 2012. Perhaps this will fare better.

I always thought Resolver was a brilliant idea, but they never gained any traction against the heavy weights (Excel).

Edit- Add link to Resolver One wikipedia article https://en.wikipedia.org/wiki/Resolver_One

Can you make this in your spreadsheet software:

http://www.quertime.com/article/arn-2012-08-22-1-25-drawings...

Do you think named cells and ranges in Excel / Google Sheets gets at this problem?

Misspelling Dan Bricklin's name within the first four words gives me low expectations…

Spreadsheets can be a pain, but be careful what you ask for to replace them...

Having contracted at two big american banks (cough jp, cough boa) that decided to build their own proprietary technology sinking ship solutions (cough athena, quartz ), it is a horrible experience.

The business get sold on the idea of the "benefits" of this new "solution" so they pay lots for it. Lots and lots. Then all tech projects are guided towards it, otherwise it becomes a "political" problem if you don't use it, even if you can present a technical case.

Basically, it works, just about, if you are building the equivalent of a simple option spreadsheet pricer. Even then, the effort required should ring alarm bells. Still, job for life for those that maintain it...

Spreadsheets somehow hit a vein with the public, so that tells you something. I've been kind of obsessed with the Tup build system, and in trying to distill what it does into one sentence, I find it helpful to say, It's like a spreadsheet for files, where the formulas are shell commands. Even though I can grok FRP on its own, I still find that analogy helpful.

Exciting stuff. I'm a financial analyst and I spend most of my time in Excel because I'm not about to send an R script to our executive team. Everybody understands spreadsheets.

But spreadsheets are terrible at everything except their original use case (small bore financial modelling).

Right now I'm transitioning a lot of my workflow to MSFT's Power BI tools, but I'd love to have a justification for using more R at work. Stencila could be that.

One project I've recently been wanting play around with is Beaker, which touts itself as a polyglot data science notebook tool. The pitch is that you can use the right tool for the job, no matter what questions you want to ask next to each other. It looks like an iPython notebook on steroids.

http://beakernotebook.com/ https://github.com/twosigma/beaker-notebook

Disclaimer (or not): no involvement, but looks very handy.

This kind of makes me want to brush off POI and attach it to Scala to develop a git-able way to create spreadsheets.

Create your spreadsheet, import it to an IDE, and create a way to intelligently create reports that can be connected to business intelligence suites... Someone must have done this.

very well done! a plaintext spreadsheet. I think it has a lot of appeal, and for all the reasons outlined. Looking at it raw, is fairly understandable. Awesome! Coolest thing i've seen all month on HN.

Awesome.

Since that title is baity, we changed it to something more neutral in accordance with the HN guidelines:

https://news.ycombinator.com/newsguidelines.html

If anybody suggests a better title, we can change it again.