Sure, I did say "typically". Obviously the form factor does not define an HSM, but in general the term is used to refer to dedicated machines designed for high performance at scale with a large number of users, rather than a SmartCard/USB device which is, typically, for a single user.