There's an inverse corollary that the lossiness of storage and the sensitivity to temperature, which seems to be a critical aspect of neural nets, may lead to a morphogenesis: the development of useful appendages for the larger cybernetics. For example, as the structure of biological organism development depends on what doesn't grow, and therefore leads to separated limbs from a body, so modules may develop as models are deeply embedded in large systems.
The training data "paradox" as it's developed by this article comes down to a quip about AI systems programs forgetting how to design and develop these systems. But this interestingly puts us on the edge of a new kind of understanding of life: how do humans, which must bootstrap themselves to exogenous knowledge every generation, manage to recover and further advance engineering? That we're constantly dropping dead seems like a huge impediment.
And there's an nascent epistemology that deals with this, which begins with the observation that humans have such limited experience maintaining a world we've built that we may figure out how to sustain our world and are already on the precipice of a collapse due to unsustainable dependencies. But this easily gets weird. The Sun will eventually explode. Meanwhile, Sol supports our world, as far as humanity is concerned, in the form of a genuine perpetual motion, regardless of the laws of thermodynamics. Isn't it the case that today's laments of AI destroying work is literally a pathos of underemployed computer nerds who can't believe that their profession would be born and ascend to the top tiers of professional prosperity then be thrown out like buggy whips within a single lifespan?
To paraphrase Woody Allen riffing about attendance for an Alice Cooper concert in the 70s: maybe the training data paradox just a case of bad vibes.
