Invoice related things were not the primary time drain since I paid the accountant to chase invoices however late invoices where a major concern because underlying cash flow is what makes or breaks you.

The other thing that becomes an issue is the classic "project management" issue, doesn't matter how good your process is if the people on the other side simply don't get it and that causes a lot of problems (projects stall because you can't get the data/information you need), fixed deadlines loom close and you are stalled waiting for information.

In the end if you care about producing high quality work it's incredibly stressful and frankly after 4 years I'm sick of it, I'm really looking forwards to working on internal systems for a big company, replacing "every client is a boss" with "this is your boss" sounds lovely to me.