Yeah, you're not allowed to do things that are specifically spelled out in the ToS, how is this surprising? Of course you don't get "unlimited access to do whatever you technically can", APIs never worked like that, why would they suddenly work like that?
When you say "you don't have the right to do what you want with the API Key" it makes it sound like specific use cases are disallowed, or something similar. "You don't have the right to go against the ToS, for some reason they block you then!" would have been very different, and of course it's like that.
Bit like complaining that Stripe is preventing you from accepting credit card payments for narcotics. Yes, just because you have an API key doesn't mean somehow you can do whatever you want.