I'm in a townhouse 4k from the centre of one of the big 3 CBDs and can't say I've ever had a problem with inconsiderate neighbours, though it's definitely possible to hear them through the walls from time to time. As for noise, the nearby (<500m) outdoor festival venue has definitely been a source of it a few times but surprisingly we heard nothing from the last few shows, probably something to do with wind direction and stage orientation. It's vastly quieter than anywhere 4km from the centre of many other big cities around the world would be (Rome in particular comes to mind - I remember almost nothing about the place we stayed in except that there was virtually never a moment's silence, ever, and it was a good 5km from the town centre).