I don't necessarily disagree with you; however, strictly for sake of argument, a couple of potential reasons:
- Providing a sense of security strikes me as a core role of society.
- People who are afraid tend to be willing to resort to draconian measures like NIMBYism to protect themselves.
- Anxiety that things could change so much that one is unable to follow their chosen way of life strikes me as a reasonable fear.