The US divides it's police by town, not by state.
And France and Germany are each twice as populous as the most populous US state.
So you're not really correct.
There's absolutely no reason why having a single bureaucracy wouldn't work well.
It actually seems like it would be much easier to have a single bureaucracy.