>it’ll be great when llms have, by default, a bit of persistent memory to remember things like this

ChatGPT already has this (not sure if it's Plus-only). In your user profile, there's a "Custom Instructions" entry where you can give it persistent prompts.