Hehe true, SSH traffic is so characteristically obvious that the packet size and timing can be used as a side channel to leak information about a session.
Tangential: but I recall reading about a similar technique used on SRTP packets to guess the phonemes being uttered without needing to decrypt the traffic.