Available on Google Cloud: Intel Optane DC Persistent Memory (opens in new tab)

HANA was in the first instance a in memory database - at least partly invented by SAP. Because SAP marketing pushed "in memory" so much, all of the SAP employees believed that HANA was the solution to all problems with SAP software. Therefore SAP named all of its products SAP HANA .... Like SAP HANA Cloud Platform, SAP HANA Cloud Integration, etc. Those products were not even related to the in memory database. It's all marketing bullshit and SAP has even taken a few steps back as too many customers were disappointed.

Ultimately I think the HANA database is an overhyped in-memory database. The reason they make money with HANA is that they are successfully replacing Oracle Databases for their SAP on-premise customers.

I have no practical experience with SAP HANA. But I’m a sysadmin and I’ve been to conferences where SAP has had stands and I have a tendency to ask too many questions.

SAP HANA is an in memory database. That’s all it is. It’s similar to Qlikview if you’ve used that. It’s licensed per socket, so scaling up is better than scaling out.

SAP will also host HANA for you, for a fee.

See. The thing is: SAP is a company for HR departments. The kinds of departments that aren’t technical, do not have technical staff to develop thing for them. (It’s an example) but your HR department talks to SAP, they deliver some service, sometimes something that the barely tech-literate members of HR is able to build on and then suddenly your sysadmins have to support those things in perpetuity.

Well. They market themselves as this anyway. And I have experience of our HR department buying all kinds of stuff from SAP and their child companies. (Concur, for example).

So I understand your incredulous-ness. This company is not made for us. And most of their tech isn’t either.

Don't know specifically about HANA. But, any database with high throughput would benefit from high bandwidth PCIe non-volatile storage. Databases with significant random accesses in their workload would see a large improvement in performance due to Optane. Nothing else apart from main memory matches Optane with its insane random access performance.

ERP applications which use HANA typically do have significant random accesses in their workload. It is due to this reason that it was initially marketed as an in-memory database. The random access performance of RAM was necessary for high performance. If HANA was just being used in in-memory mode, the speed increase is probably not that impressive. But, if there is non-volatile storage involved, then yes the speed increase would be impressive.

Couldn't tell you what it does, I only know that the advertising of it in airports is ubiquitous. In the dozen plus airports I've been in this year, they've all been plastered in adds for HANA.

I've used SAP HANA for about 5 months earlier in the year. Between the overselling marketing, the expensive cost of the thing, and the bugs + growing view that SAP are using customers as guinea pigs; I'd say it's like IBM Watson for data.

You can get good performance out of it, no medals there as any IMDB would give you that.

My pain points were:

- it is sold as the virtualisation layer to rule them all, until you find crazy breakages. SAP's SQL dialect is case sensitive. That'll frustrate a lot of downstream databases that you deal with, or the users thereof.

- virtual warehouse. I'm not a Kimball expert, but our resident experts came to the same conclusion as I did, that you can't really build useful VIRTUAL data marts directly from disparate downstream data sources without persisting that somewhere. That then defeats the purpose, in my quasi-professional opinion.

- I spent over a month looking at the SDA/SDI to "Big Data" parts. Very immature, found interesting instances where push-down of predicates were inconsistent and poor. It's one thing to say you've got Apache Spark integration, but it's another thing when your system can't figure out that data comes from the same system, and this you could do some expensive computations downstream instead of saturating the network pulling a lot of data. I did my work early this year, so I would expect improvement here. The problem though was that many of us started feeling like SAP are releasing a product that should spend more time in development before coming to us. I'm entitled to say so because they charge an arm and a leg.

- data modelling/development. SQL is Turing complete, but some things are still hard to do in it. Instead of giving us more capabilities when modelling our data, they're getting "distracted" by wanting to tuck marketing boxes like R support, NodeJS support, etc. I didn't spend enough time on those to feel comfortable with producing a fair opinion, so I'll reserve one.

- SDLC, security. SAP has solid products there, and I learnt quite a lot around security integration. There were some quirks, but those were largely because of our requirements, not SAP. I enjoyed working on data access and security patterns.

- IDE. They initially used a modified NetBeans, which was also available as a plugin. I'm not a NB person, but this was solid. They decided to move to a web-based IDE. I don't know why, but they deprecated certain things while their web thingy was not even feature complete. It was a growing mess when you'd find that you can only do something in X but not Y.

To recap, a high cost for performance that you could get from a competitor, or even a well baby-sat Apache Ignite cluster (for the features they support, at least they aren't lying in their marketing). I think of it as a lame caching layer, that's useful if you're already invested in the SAP ecosystem, think of SAP CRMs or even their warehousing as a stretch.

So, no; it's not revolutionary in my view. The "specialised hardware" is marketing bait, our IT (I too was) were convinced that one could build themselves the same hardware as was speced. When I first heard of the special hardware thing, I thought of a custom Intel with features that only SAP would have, DDR4 RAM at crazy clock speed, etc. Only to find that it's a sly way of controlling how much they charge you. "Want to add moar RAMs? Pay us so you don't vid warranty, and we'll charge you more of course for the extra RAM".

To conclude my rant, I hope you've found my view useful. My conclusion when I rolled off from my SAP HANA (notice how little I've used this acronym, I despise it) project feeling like it was making everyone in the room bad. People were forsaking their Oracle DBs and Hadoop clusters to make design compromises for an over-marketed database thingy.

I'm curious how endurance is handled in this case. Is there a wear level guarantee when provisioning it? I guess this is applicable to any consumable though this seems to be a pretty expensive one.

> The persistent part is that it's persistent across reboots.

So does that mean it can be considered persistent like a regular SSD/HDD?

How much of a problem is the gap for SSD's? Don't those have buffers with enough battery to write out if they lose power?

This is byte-addressable persistent memory. They look like DRAM DIMMs and they plug into DIMM slots. You access them using your memory controller and not your storage controller. People sometimes refer to them as non-volatile memory (NVM). Intel used to call it Apache Pass.

They're a nightmare to program because OSes do not have a good abstraction for them (at least not yet). Accessing them through the file-system seems sub-optimal (this is byte-addressable memory and not a block device). Accessing them through virtual memory is also pretty bad because they're much slower than DRAM.

It uses Intel Optane. It's basically super fast SSD used as RAM.

It's between RAM and SSD. Much closer to RAM speeds but data persists after turning off power. Biggest advantage over just being a faster disk is that it's byte-addressable random access like memory, so you don't have to deal with writing pages/blocks to drives.

The throughput and latency is fast enough that many apps can skip using memory and work straight from this. We already see this in some mobile devices that only have solid state and instant boot times because the data is always ready, with no need to shift from disk to ram.

It'll be awhile before OS and applications take advantage but it has potential to be a major shift in how storage works.

Disclosure: I work on Google Cloud.

NVMe is (now) a more general purpose "speak to flash like things". This Optane memory stuff is made of "flash", but unlike our Local SSD offering (or AWS's i3) it's at a latency and throughput closer to DRAM.

Despite it being marketing material, I find the pyramid diagram [1] helpful. This blog post is about "Optane Persistent Memory".

[1] https://newsroom.intel.com/wp-content/uploads/sites/11/2018/...

A better comparison would be EC2 x1 instances which are high memory instances designed for SAP HANA like these ones: https://aws.amazon.com/ec2/instance-types/x1/

The benefits of persistent memory, as I understand it, are principally improved performance (higher throughput, lower latency) and secondarily ease of programming. (Edit: another obvious benefit is cost per gigabyte; this stuff is cheaper than RAM.)

The benefits of having access to that technology on GCP (or another cloud) are the usual: reduced operational burden, increased availability, flexible pricing structure, elastic scalability, etc.

Your question about pets vs cattle is a non sequitur. Nothing about this announcement or the underlying technologies are suggestive of how they should be used (or misused). It's just a new tool in the toolbox—and as a distributed systems engineer specializing in database technology, having easy access to this hardware at scale is extremely compelling.

too bad you got downvoted, i would imagine in-memory DB and large cache.

Can't you already do this with an OODB? I'm asking sincerely, since I have never quite understood why OODBs aren't used more often.