The difference here was, he didn't say "it was just a joke - you don't need to be offended" - which is a definite non-apology.

He's saying "it was intended as a joke", but then goes on to deliver a sincere apology, taking responsibility and saying that there's no excuse for his behavior.

Ever seen a web developer rant about how his client insists on a project's price, because his third-removed nephew could implement it in a weekend? Or a web designer having to use some gaudy color-palette, comic sans, animated gif, or feature like an on-page pageview-counter or marquee? I see it every day, and there is never some voice of opinion telling them to not vent.

The developer, designer, and this CEO can very well want the job or position. They want to make the client happy, but they are going to have to vent, at some point. Perhaps there are better ways of doing it, but in a sense, the vents are "a joke" and don't negate the fact the customer is still in good hands.