On Linux you can set this in /sys/class/power_supply/BAT?/charge_{stop,start}_threshold – you can read and write a percentage (1-100) to there. You'll have to set it on every boot.

I assume there's some way to do this in Windows as well, but I don't know it.

At least one of those; depends on the device. As sibling comments note, plenty of devices expose it in software, where in Windows you use the vendor's utility and Linux puts it in /sys, but I've got laptops that either don't do that or I couldn't find it but I could configure it in the BIOS settings. So you might have to dig around a little.

Lenovo vantage, under battery management or something like that. Has been an option for quite long.