The compromise here could be you can now have vacations in Manhattan. There's an odd thing that happens to most of us where we become convinced we HAVE TO live in certain places and we get tunnel vision because of that. Personally, I think the sweet spot is living 20-40 minutes of such places. It tends to be more peaceful further from downtown areas, considerably more affordable, yet still close enough you could day-trip it and enjoy the amenities.