What you really want for time-series data is a column db such as cassandra (or vertica etc.), perhaps HBase, perhaps a RDBMS, or perhaps a plain old log-file.
What you most definitely don't want is Microsoft Access or MongoDB. Thinking about it, MS Access might still work to a degree.
To expand on the parent, a column store like HBase or Cassandra is perfect as each row can represent a timeslice and then each column can represent a single event or record within that timeslice. As the row gets evicted from it's initial storage in memory, it will be written to disk contiguously, in sorted order, and batch reads of this data is sequential.
It is possible to use MongoDB to store a timeslice as a document, but it is not designed to scale out to store very large numbers of columns within a single document.
What do you mean by "very large numbers of columns" ? I've seen some mongo users with very rich, i.e. large, document models.
Vertica was not free, HBase was really heavyweight, & I knew i didn't want RDBMS. The sensor data itself is plain csvs ;)
Cassandra still does look interesting, although it looked like it would take me much longer to get it going; perhaps it will get rewritten. (The mongo solution only took a week to get something usable.)
This might be poorly titled: it's not just about the storage as it is about the aggregation of disparate sensor data into coherent, continuous data streams.
Temporal Collection { _id: "X1", data_temporal : [ { time_start: SomeDate, time_stop: SomeDate, _id: "ID2" }, { time_start: SomeDate2, time_stop: SomeDate2, _id: "ID2" }]
Data Collection { _id: "ID1", parent: X1, data: { field1: "some info", field2: 34 }, _id: "ID2", parent: X1, data: { field1: "Some info new", field2: 34 } }
What is cool about this is that if you have access to the data like ID1, you can easily find out when it was added and how it changed.
If you have access to the temporal ID, X1, then at any time you can see what the data looked like.
If you need to relate data, the "foreign key" used is the data_temporal ID. In this way, it is possible to ask what your key value store data looked like at any time.
But, this could be off from the article.
This also works quite well in a relational database.
I would love to have a look at their Arduino based solution.
