I think a lunar colony would be great but helium-3 isn't a good reason for one.
If you can get net energy by fusing He3, you can also do it by fusing deuterium, which is easier. And the end product of deuterium fusion is He3! (Half the time directly, the other half producing tritium which decays to He3.) So instead of sifting through millions of tons of dirt on the moon, you can just make He3 on Earth and gain energy in the process.
Deuterium fusion produces neutrons but they're lower energy than D-T neutrons. Fusion startup Helion (funded in part by YCombinator) is working on a hybrid D-D/D-He3 fusion reactor, and says only 6% of the output energy would be as neutron radiation.