In the same vein as "you can't go home again" (you can go back to the physical location, but you have changed, anybody living there has changed, the buildings have changed etc, to the extent that it's no longer the same thing as your internal conception of "home").. "you can't start from scratch again".