No, these NLPs aren't idempotent. Even if you ask ChatGPT the same question multiple times you will get different answers.

LLMs calculate a probability distribution for the relative chances of the next token, then select a token randomly based on those weightings.

They inject randomness in a layer were it does have small impact on purpose.

Also to give it a more natural feel.

Can't find we're I read about it