I'm a PM at Microsoft that worked on open-sourcing PowerShell. As Windows PowerShell, it's a built-in Windows component, and the number of assumptions that could be made because it was both closed-source and part of the OS were
immense. Even getting it to
build outside of the rest of the operating system was crazy hard. Then figuring out to install it in a supported way was hard, and only
then did we get to start figuring out how to eliminate usage of private APIs and start doing legal reviews.
And we're "just" a language runtime and a shell shipped in the OS. I can't imagine how hard it would be to unwind the proprietary bits of something like a GPU architecture running in a tightly integrated SoC with multiple vendors who are all deeply protective of their IP.
I say this all as someone who has been a proponent of open source for 15 years. And all the work was absolutely worth it, and deeply rewarding from a personal perspective.
But I can't say it would probably make the same sense for Nintendo to go through that effort with Pokemon or the N64 architecture.