The main issue is that the image needs to have a high enough resolution to be sharp at all zoom scales. Currently my images are vector graphics that I rasterize depending on the screen resolution.

The Escher Print Gallery requires even larger scales as it uses a zoom factor of 256 across the image (vs 16 for my images)

Others have solved this by either vectorizing the Print Gallery or even rebuilding the scene as 3D signed distance field that can be sampled via ray marching.[1] The later yields the best result but I did not want to copy it.

[1]: https://www.shadertoy.com/view/Mdf3zM