This is a reminder that these islands were uninhabited prior to European discovery.

It is true that they imported ... basically slaves ... from/via semi-nearby islands to work on it, but it's not like it was some ancestral island to them. When the work stopped, they were returned to the islands their ancestors came from (or at least via).

(This case is somewhat different than the also-originally-uninhabited Falkland Islands, where most people living there were always of European descent).