Having been messing around personally with getting my own blocks of IP addresses and routing[1] - I've become terrified at the idea of implementing access control based on IP address.

Unless your own organisation in the RR has the IP addresses assigned to you as Provider Independent resources, there just seems to be so many places where 'your' IP address could, albeit most likely accidentally, become not yours any more. And even then, just like domain names, stop renewing the registration and someone else will get them - I was that someone else recently...

[1] AS202858