But you have no freedom when contacting my servers. I can send you 403's just because I don't like your face. There is zero entitlement that you have any access to my servers in any way I don't permit. If I say no automated access then on what grounds to you have to do it anyway?
> Why is Google the only one who can do it?
Because site operators explicitly allow them automated access. If you want the same treatment you have to ask for it.