Mid-tier cities give a lot of bang for your buck -- Salt Lake, Phoenix, the midwestern midsized cities (Cleveland, St. Louis, Indianapolis, Kansas City, Pittsburgh), Atlanta, etc. Even Portland compared to other coastal options.

And even in some of the "very expensive" housing markets, you can still find homes that are affordable relative to income within 40 minutes of downtown areas. Chicago and Boston both come to mind.

The primary differences, as far as I can tell, are that:

1) Niche senior positions are harder to find in cheaper cities (think "deep hard tech expertise"). For example, nearly all of the major corporate research labs in CS (MSR, Google Brain, Google Research, IBM Research, Oracle Labs, ...) are in or around expensive CoL areas. On the startup side, "hard tech" startups are very often more capital-intensive so are even more attracted to geographic VC bubbles.

2) Moving up within BigCos is sometimes more difficult if you're in a satellite office.