I agree that there are situations that not-infrequently arise that require additional IP port access restrictions. However, do you agree that removing the notion of privileged ports [0] would
1) Not give you the more-fine grained access restrictions that you're looking for
2) Actually weaken security on Linux systems
? (Additionally, GRSecurity, SELinux, and -apparently- AppArmor all appear to provide the finer-grained control that you're looking for. There is also this [1] which lets you do something like what Windows Firewall does and use iptables to restrict which uid/gids can do certain types of IP communication.)
[0] That is to say, remove the restriction that one must run as root to bind to ports < 1024.
[1] https://www.debian-administration.org/article/120/Applicatio... [2]
[2] Even though the kconfig option has changed names, it appears to serve the same function (see the Owner section of [3])