I've always assumed "giga" is capitalised because it one might confuse a lower case g for "grams". Technically, 1gm is one gram-meter (grams * metres), whereas 1Gm is a thousand kilometres. You don't often encounter weird combined units like gram-metres outside of physics, except maybe for kWh (kW * 1 hour), but I don't see why you wouldn't be nice to physicists and give them the extra clarity.

Other order of magnitude indicators are capitalised to distinguish them from their smaller counterparts (millimetre/Megametre).

Wikipedia states that the distinction is because units named after a person always start with a capital letter. I've always assumed Hz has to be capitalised to prevent confusion with the hecto prefix (hHz being 100Hz). I don't think there's a unit or order of magnitude that's abbreviated to `z`, so in theory "Hz" could just be "H",

I think Hz is more readable, but that's probably because I'm used to it.