The only case that I know of when manual memory management is slower is short-running programs which exit before the first GC run, therefore removing the need to do any deallocation at all.

Many times it is less resource intensive to do manual memory management, it just depends. The sacrifice in performance comes more from runtime features like type checking.