undefined | Better HN

0 pointstptacek1y ago0 comments

IIRC, (a) and (c), which I think sort of implies (b)?

Really what we'd have wanted to do would have been to give Fly Machines MIG slices. But to the best of my understanding, MIG is paravirtualized; it doesn't give you SR-IOV-style PCI addresses for the slices, but rather a reference Nvidia's userland libraries pass to the kernel driver, which is a dance you can't do across VM boundaries unless your hypervisor does it deliberately.

0 comments

transpute1y ago

Hypothetical scenario for Nvidia licensing of fast-start microVMs:

1. Instead of blocking VM start for license validation, convert that step into non-blocking async submission of usage telemetry, allowing every VM to start instantly. For PoC purposes, Nvidia's existing stack could be binary patched to proxy the license request to a script that isn't blocking VM start, pending step 2 negotiation.

2. Reconcile aggregate vGPU usage telemetry from Nvidia Fly-wide license server (Step 1) with aggregate vGPU usage reports from Fly's orchestration/control plane, which already has that data for VM usage accounting. In theory, Fly orchestration has more awareness of vGPU guest workload context than Nvidia's VM-start gatekeeping license agent, so there might be mutual interest in trading instant VM start for async workload analytics.

tptacekOP1y ago

Sure. We also could have virtualized CUDA ourselves, used MIG on the host-side, and done a proxy PCI passthrough driver thing in Cloud Hypervisor. I think we could have gotten it to work. But it would have been a huge lift. I'm glad we didn't try.

transpute1y ago

> proxy PCI passthrough driver thing

Do you mean vCS [1], which is already integrated and licensed by KVM/RedHat/Nutanix, Xen/Citrix and VMware?

It's distinct from SR-IOV, PCI passthrough, vGPU-for-VDI, and MIG.

[1] https://blogs.nvidia.com/blog/virtualcomputeserver-expands-v...

1 more reply

j / k navigate · click thread line to collapse