Some good prompt-reply interactions are probably fed back in to subsequent training runs, so they're still stateful/have memory in a way, there's just a long delay.
State is a function of accumulated past. That does not mean that having some past written down makes you stateful. A stateful thing has to incorporate the ongoing changes.
