Visiting Billingsgate will teach you that quite a few restaurant folks (and quite a few "normal people") shop there, in addition to (smaller) fish traders.
Smithfield, at least to me, always had a different vibe; more pro-to-pro/business-to-business, but also they deliver or commission to pre order there; different from Billingsgate.