We often think of 'Tea' as a very English thing (and it is in many ways now), but of course it has it's roots in China/Asia i.e. global trade.
The Japanese borrowed ideas from elsewhere for quite some time (as has everyone of course), but they would view 'building Western warships' as something very different from 'importing millions of Westerners'.
Also, one could argue that Japan, for a variety of reasons, was far more isolated than other places.