I'm not sure you have the same definition of "cheap" as I do. $5/mo for a 1GB DigitalOcean droplet is cheap.
Buying old-but-badass hardware seems to be similarly cost-effective for that purpose (if that link is to be believed), but where (other than "a friend in ops") can you get 1U/2U of colo hosting for anywhere near that price?
I guess, if you're talking about the marginal price of an extra 1U/2U on top of your existing production rack, it can be negligible. For non-business-affiliated goofing-around, though?