Yes, that is roughly correct. You nay want to look up "reversible computation". It's a fundamental part of quantum computing, for one thing.

The key insight is that a (finite) discrete, reversible system will always eventually cycle back to its original state. This fact has very interesting follow-on implications for the concept of entropy and the Second Law. If it is guaranteed that a system will return to a prior state, how can it also be true that entropy (disorder) always increases?