It is. If you really want to get after the password scans, iptables and connection tracking. After N number of new connection requests, drop for X minutes.

Any IPs that are trusted, put ahead of that logic.