Show HN: Shadeform – Single Platform and API for Provisioning GPUs (opens in new tab)

Now, regarding the product - this is amazing. From both the perspective of saving time and money digging through providers to the part that actually I find the most impacting - the simplification of the AWS console mess to a niche use case. While I understand GPU's are the hot thing now and there is a scramble for a single Flop, if you ever decide to pivot, I'd gladly pay more money each month to use such a simplified niche AWS/Generic cloud console.

Can't wait to have a chance to play with this more, keep up the good work and good luck!

The single interface for any cloud GPU is cool, but hard to imagine it taking off without some additional features.

I think for lots of shops the hardest part isn't the compute but moving the data around. Ie for us, we use s3, some lustre caching and spot instance nodegroups. We are a deep learning research team that spends roughly 40-50k/month on aws compute for training jobs. I imagine this is somewhat mid tier, maybe more than some but certainly far less than others.

For inference, data egress costs could be less of an issue, but your service would really need to be almost invisible. It probably would be pretty complicated for a number of reasons, but if you could design a "virtual on-demand nodegroup"™ that I could add to my existing clusters and then map to whatever k8s stuff I want, that would probably be useful. I would need to be able to auto deploy a base image to the machine and then provision the node and register with my cluster.

Just some unorganized thoughts. Good luck and have fun.

Provisioning: https://www.youtube.com/watch?v=7WyKPMS80Pk

Reservations: https://www.youtube.com/watch?v=Ab5GmfMYWKA

- the people who thrive at this use orchestration, like Slurm or Kubernetes. So the nodes I buy should join automatically to my orchestration control plane.

- people who don’t use orchestration or don’t own their orchestration will not run big jobs or be repeat customers. It doesn’t make sense to use nonstandard orchestration. I understand that it is something that people do, but it’s dumb.

- so basically I would pay for a ClusterAutoscaler across clouds. I would even pay a 5% fee for it automatically choosing the cheapest of the fungible nodes. I am basically describing Karpenter for multiple clouds. Then at least the whole offering makes sense from a sophisticated person’s POV: your Karpenter clone can see eg a Ray CRD and size the nodes, giving me a firm hourly rate or even upfront price to approve.

- I wouldn’t pay that fee to use your control plane, I don’t want to use a startup’s control plane or scheduler.

- I’m not sure why the emphasis on GPU availability or blah blah blah. Either AWS/GCE/AKS grants you quota or it doesn’t. Your thing ought to delegate and automate the quota requests, maybe you even have an account manager at every major cloud for that to bundle it all.

- as you probably have noticed, the off brand clouds play lots of games with their supposed inventory. They don’t have any expertise running applications or doing networking, they are ex crypto miners. I understand that they offer a headline price that is attractive but for an LLM training job, they “vast”ly overpromise their “core” offering.

- if you really want to save people money on GPUs, buy a bunch of servers and rack them and sell a lower hourly rate.

My co-founder and I always joke that there are only two hair-on-fire problems in 2023 and they can be summarised in 6 letters: GPU & PMF.

Really love what you're building.

You'll quickly find that your platforms primary use is to turn stolen credit cards into cryptominers.