Safe browsing doesn't include sites for encouraging hatred and violence, etc. Only malware, social engineering, and "harmful"/"unwanted" applications. If they start including those sort of sites in their safe browsing lists, that would make your point here more relevant.
(Of course, some people get hit by safebrowsing unfairly. But I think in most cases, it is because someone compromised their site and used it for a malicious purpose, and then they struggle to get Google to remove it within a timeframe which is reasonable.)
