Web crawlers generally allow sites to remove them from the index.
Are there any crawlers used for commercial purposes which refuse to remove sites from an index if they ask? The distinction from OpenAI is that there is no way to be removed from openai's training set.
You can remove yourself from the crawler not but not from what they previously crawled.