In this theory the reconstruction filters need to work as both downscaling and upscaling filters. A box filter looks good for integer-ratio upscaling, looks bad for non-integer upscaling, and looks terrible for downscaling.

What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale.

> Or, you could say, the smaller I want filter regions to be.

And that's the opposite of you get from linear filters. There's a resampling filter called sinc that's "perfect" - it reproduces every frequency in the input signal - but it looks terrible for images because it tries to reproduce pixels based on every other pixel in the image. So you get ringing all over the picture instead of sharpness.