I'm all for cross compilation, but in reality you still need running copies of those other operating systems in order to be able to test what you've built.
Setting up 3 or 4 VM images for different OSes takes a few minutes. Configuring 3 or 4 different build environments across as many OSes on the other hand ...
Sure, but building typically takes more resources than executing, so it's not really feasible to use a Raspberry Pi to build, but it can be for testing.
