I'm not a cyclist but I've heard similar complaints from cyclists about the area. The downtown is getting more bike-friendly with dedicated lanes here and there, but - as you said - the mountains are the real appeal there.
It's something we need to work on.
Sorry you didn't have a great experience!