- the compositor insists of doing (with interpolation, depending on the compositor you use the results do hugely differ in quality)
- scale to "exactly" that fraction.
But if I "as a normal user" ask for a scale factor of 1.33 I do in general not mean "scale exactly to 1.33", neither "scale exactly to 4/3" but what I mean is "make it roughly 4/3 larger as 3/3 is to small and 6/3 to large".
And a lot of program can do that just fine, especially wrt. fonts.
Like e.g. a terminal which is set to font size 12 on 3/3 would just scale the font size by 4/3 and then round. Result is a crisp font roughly 4/3 larger which fulfills all requirements. Only drawback is that:
- it needs to be handled by the application, but so does any HiDPI support
- due to rounding the ratio of font sizes might be of by 1 pixel, irrelevant on a HiDPI screen
- this doesn't help with pictures or other layout, but even they can be handled better by the application then the compositor(1).
Wayland should just give applications the choice to to handle fractional scaling themself, like it does for HiDPI scaling. Any mainly text based program can support that reasonable well (terminal, editor, etc.), so can many games. Also due to HiDPI screens becoming increasingly more common (e.g. 1440p on gaming laptops) it also becomes more common that programs come with a layout/style/prictures for that, in the same way they have one for 4k.
Just to be clear a program should be able to opt-into "fractional scaling hidpi" mode based on the specific scaling. And I think it also would be fine if wayland or DEs doesn't support arbitrary fractional scaling but just a set of common scaling factors like 11/10, 5/4 (for 4k), 4/3 (for 1440p) maybe 3/2, 7/8.
(1): The compositor is in generally the place least well equipt for doing any scaling. As it only knows the final result "as image". But UI/Layout scaling is not the same as scaling an image. Which yes, doesn't work well for fractions.