Now see,
that's why everybody should use Tor now and then! :) That way this unique characteristic will become common enough for a generic filter on Tor users to become ineffective (by their measures). If you don't want to be seen connecting to Tor directory authorities and to relays directly, use a Tor bridge. Better yet, run one yourself! If you are using obfuscated bridges, you can (as of now) fool even pretty sophisticated DPI boxes which won't be able to fingerprint your traffic. (cf. continued Tor developers' battles with China's intense DPI infrastructure)
https://cloud.torproject.org/
https://www.torproject.org/docs/bridges#RunningABridge
Honestly though, "Tor user" will probably remain an interesting / valuable (by their standards) enough heuristic for a long time to come.