Honestly the lore of w40k is quite fun to read, if you’re into dystopian and fantasy sci-fi.
It seems like both sides are fine to have them be reconciled, but it's an important narrative gadget that can be used to get humanity to fight itself in-universe.
Also interesting is that humanity's "lost" technological progress seems to eclipse some of the other races in the W40K narrative, with even the Tau (space dwarves with robots) and Eldar (space elves with crystals) freaking out when humanity brings giant robots, because the sheer physical impracticality of a gigantic human shaped robot is noted, with nobody aware of how they continue to work.
BattleTech had something similar: it was considered cheaper to keep replacing humans than to replace the mechs, because hardly anyone knew how to repair or build mechs.
Scientists only get to talk to the public every 100 years or something, wasn’t it?
IT was never allowed to talk to scientists.
Seemed like a modernist idea, even at the time of publishing.