undefined | Better HN

story

0 pointsiforgotpassword8y ago0 comments

Well technically you could just defragment your physical ram to free up contiguous memory, this is also what Linux does with thp enabled.

But, unless the analysis in the OP is wrong the problem stems from Windows handing out freed huge-pages again before zeroing them, so whichever program gets the page the second time might write data to it and then have it wiped a moment later, which sounds like a plain bug to me.

0 comments

dragontamer8y ago

I don't think THP is a good example, because its not considered very stable actually.

https://oracle-base.com/articles/linux/configuring-huge-page...

> That sounds like a good idea, but unfortunately Transparent HugePages don't play well with Oracle databases and are associated with node reboots in RAC installations and performance problems on both single instance and RAC installations. As a result Oracle recommends disabling Transparent HugePages on all servers running Oracle databases, as described in this MOS note.

-------------

Anyway, the very point of huge-pages is to gain ~3% to 5% faster program speed by keeping more of the memory-management units of the OS inside of the TLB-cache of the CPU.

Running a background thread to defragment the physical memory system of your computer WHILE your critical tasks are running... very most likely kills the performance benefit you were trying to get.

Linux's raw Hugepages work fine, but have their own set of drawbacks as I described in the previous post. For the moment, it seems like a better idea to use "normal" Linux Huge Pages than to use the THP. Indeed, I'm seeing reports that some people get 10s of millisecond pauses when their application does a 32-byte malloc with THP enabled (Linux Kernel decides to garbage collect + defragment + turn your pages into a Huge Page). A lot of programs don't expect a tiny malloc to cause such a major performance hit randomly (ie: video games).

https://groups.google.com/forum/#!topic/mechanical-sympathy/...

https://access.redhat.com/solutions/1560893

---------------------

Best practice (in both Linux AND Windows) is:

* Code your program to accept Large Pages if possible. But consider the failure case, and fall-back to normal 4kB pages if you fail.

* If performance is CRITICAL (ie: servers), then allocate the large page at the beginning of the program, and NEVER LET IT GO. Start the program early when the OS boots up to ensure that Large Pages are available.

* Linux's alternative is to have the human / SysAdmin manage large pages manually. Divy them out to the programs as needed. Which works just fine, although its a bit of a hassle.

j / k navigate · click thread line to collapse