Its not so much that smalltalk is an OS , but rather that is a Live Coding OS. The ability to hack application while they run, is something that could have a profound effect on user experience. Such features are hard coded to apps themselves and are not considered languages features for all non smalltalk apps.
But with smalltalk is not that you run an OS as much that you can hack any application on the spot , while it runs and see your changes immediately. This happens because smalltalk does not compile source files but rather methods and of course because of the flexibility of the whole system.
Smalltalk is an extremely user friendly environment for users that love to hack the easy way and not get insane in the process.
Where you could get almost a similar experience while using a strong type language, with help of reflection and by having dynamic libraries as the only form of executables.
The System 3 version with its Gadgets toolkit was quite nice.
Sadly another GUI desktop OS that didn't managed to become mainstream.