Why hasn't traditional AV created a similar product then?
The way these "next-gen" endpoint systems work is by doing a deep analysis of every file, and like you said, uploading the hash to a central server for faster processing later. Your use case is atypical for CB customers but I fully believe you're having these issues. There's a drawback to every kind of endpoint protection. It does seem that the more general population is worse off with traditional malware protection than CB, but your use case seems non-traditional.
I should note that I am not affiliated with any endpoint product and AFAIK the absolutely massive company I work for doesn't even have an endpoint product that competes in the next-gen space. I'm just a security consultant who sees a lot of Fortune 500 companies and notices the trends they set/follow. It's trending away from Symantec and towards Carbon Black/Tanium/Cylance/etc.
(Not that it’s really fancy new tech - checksum AV have been around even since DOS. I’d have thought the best thing would be a combination of the two).
Obviously it's not perfect for everyone and technical staff will often need to run esoteric and constantly-changing applications, so whitelisting isn't always possible. In that case, using a checksum and having the central server is a better way of handling it. Better yet is something like FireEye which can intercept your file downloads and scan them before it hits your machine. I can't speak for which next-gen endpoint solution works best since that's not my area of expertise, but I can say it's better than traditional AV (which is basically useless). In that case, blacklisting is the better choice, for software no one should have installed.
"Spying on employees" is an interesting take on what I consider to be basic security. I'm heavily involved in technology that, if the end user saw what we could see, they'd be horrified. Basically, if you're in the US and using your employer's laptop on your employer's network, you have zero privacy and everything you do and every site you visit is being logged into a central log repository and can be made available to the security and audit teams at a moment's notice. Most of the time no one is watching it, no one except an AI looking for anomalies and reporting on outliers, but it's possible. If you're doing DNS lookups to your company's DNS server, they know every site you've visited. If you're using telnet or ftp or POP3, they know your passwords too, because they're likely sniffing internal network traffic as well and storing packet captures. And they may even be breaking SSL at the proxy or gateway level, so that doesn't help you.
Basically, if you're worried that Carbon Black sending a list of your installed applications is your employer "spying" on you, they're already collecting far more data than you think. Installed applications is the least of your concern. But again... that's not your laptop and it's not your network. It's all owned by your company, and governed by their acceptable use policy in the employee handbook.
1. Choose an existing trust list (over 1000 apps/dlls etc trusted) or build your own in a few minutes.
2. Install file filter driver - as soon as driver is installed on endpoint, instant protection, no scanning of drives, folders etc needed.
3. User runs a trusted app - intercepted at kernel, fingerprinted and matched allowed to run. Imperceptible to user.
4. Malware tries to execute - intercepted at kernel, check to see if on trust list, not on trust list - blocked.
5. Need to add a new trusted application? One click, no rules to amend, no whitelists to push to endpoints. All endpoints inherit new app trust and app can execute across global enterprise.
From what I understand, AV just tries to match hash signatures. CB is doing more analysis. It's a tougher not to crack.
