Yeah my own Win95 machine had 32MB and ran super happy. The 4MB machine was frankensteined from very random parts for a friend who fried the PSU on their 8MB - the difference between all 3 setups was crazy, I think at 4MB you had to use a 16-color palette? At 640x480x8bpp video memory alone should eat 300KB (I don't think there was double-buffering?)
I sometimes wonder, what would be the "smallest" machine you could feasibly DIY/build today, that would still allow you to be somewhat productive (e.g. write code, very basic web browsing, SSH, etc). I recently spent a week with a PowerBook G4 (2002) and it's very usable (save for the fan noise). There's probably a lot to explore with modern MCUs.