I had the same frustrations until the other day I saw a tip to enable 'Use the new Virtualization framework' under Preferences -> Experimental Features. Since then Docker's background CPU usage has dropped to 1% and I no longer bother stopping it when I'm not using it.
I'm happy your platform of choice has no bugs nor quirks. Could you let us know which one it is? We can probably help you find the bullshit you put up with that has become second nature.
