Yes, building a memory device in a known configuration is erasing bits. Once you've built it, you can use it until it breaks. As long as you decompute the bits you've temporarily stored in it, restoring it to its original configuration, you don't inherently have to dissipate any energy to use it. You can reuse it an arbitrarily large number of times after building it once. If you want to compute some kind of final result that you store, rather than decomputing it, that does cost you energy in the long run, but that energy can be arbitrarily small compared to the computation that was required to reach it.

Consider the case, for example, of cracking an encryption key; each time you try an incorrect key, you reverse the whole computation. It's only when you hit on the right key that you store a 1 bit indicating success and a copy of the cracked key; then you reverse the last encryption attempt, leaving only the key. Maybe you've done 2¹²⁸ trial encryptions, each requiring 2¹³ bit operations, for a total of 2¹⁴¹ bit operations of reversible computation, but you only need to store 2⁷ bits to get the benefit, a savings of 2¹³⁵×.

Most practical computations don't enjoy quite such a staggering reduction in thermodynamic entropy from reversible computation, but a few orders of magnitude is commonplace.

It sounds like you could benefit from reading an introduction to the field. Though I may be biased, I can recommend Michael Frank's introduction from 20 years ago: https://web1.eng.famu.fsu.edu/~mpf/ip1-Frank.pdf