The "difficult to return zones" are exactly that: the subset of zones that are represent the greatest challenge to resettle. Most of the exclusion zones with less severe categories have been lifted [1].

The point is, people assume because of Chernobyl that a nuclear disaster will render land uninhabitable permanently, or at least for decades. That is not the case. Chernobyl remains excluded because there's no real incentive to clean it up and reopen. The city was built to support the power plant, so with the latter gone there's no real reason to put in the effort to decontaminate the former.

1. https://www.pref.fukushima.lg.jp/site/portal-english/en03-08....