Considering basically every router has the same address
They have the same IP address, not necessarily the same DNS hostname, which is what the certificates are tied to. The user would just be told to connect to the hostname (possibly printed in the sticker) rather than to the IP.
That's certainly one option, but what happens now if I change the IP of the router in its config, because I use multiple in my LAN, one as router, the others as AP?
There is no option for any of this that isn't completely messy and hacky