Even if we don't opt out, it's not plaintext data right? Further, I feel like it's likely to be the tokenized version right?
And as far as strategic plans, I don't think it regurgitates novel, one source information does it? Isn't the regurgitation more like boilerplate, seen many times type stuff?
Has anyone actually been able to prompt a password out of an LLM? That kind of thing should be ignored because it's high entropy and rare afaik. For example, my name is bongodongobob and my password=hunter17. Do you really think someone will be able to pull this fact out of an LLM at some point? That doesn't seem to be the way they work as I understand it.
