Might as well add my anecdata...it works perfectly for me.
I have an M1 Mac Mini, with an Apple Magic Keyboard and Apple Trackpad.
On first setup I had to connect them via the lightning/USB cable at which point they paired automatically. Then I disconnected them and they've been working since, through multiple reboots, via BT.
And no kernel panics either.
It does make sense that you need something wired on first setup, though it should not be forgotten on reboot. That could easily be an issue with either the peripheral or the M1.