I've bike commuted to work. In a very bike friendly area.
Everyone I know has been in an accident of some sort. Hit & runs, side swipes, doored, pushed (deliberately), etc.
After better infrastructure, the most effective way to improve safety is to have more cyclists. I think we're in a rough transition period, but things are definitely getting better.
One of my bike buddies thinks things will really change once moms start cycling en masse. That'd change the perception from gonzo bike dudes to totally uncool in a heartbeat, evoking much less hostility.