undefined | Better HN

0 pointsjhugo4y ago0 comments

Any reason for doing audio and video work on the backend rather than adopting the (much easier to scale) SFU model and just forwarding packets? Similarly for the physics, seems like doing that work frontend would scale much better.

0 comments

3 comments · 1 top-level

jupp0r4y ago· 2 in thread

Even if you use stream forwarding only, network will be a major O(n) cost factor when using cloud providers, at least for video.

There is also some super interesting middle ground between full video forwarding and selective reencoding by using SVC for cheap video resizing on the backend.

jhugoOP4y ago

SFUs aren't forwarding everything. Using either discrete simulcast layers or SVC they're forwarding one size out of several (usually 3) to each other participant.

Bandwidth will be a high cost no matter what (and anyone planning to scale up real-time video better be prepared to move off the cloud at some point) but needing compute resources for encode/decode on the backend makes large scale realtime video infeasible for any reasonable cost. The only successful large scale deployments push most compute work to the clients.

An extra decode/encode step also adds latency and the total latency budget before UX is impacted is small.

jupp0r4y ago

You can avoid bandwidth costs: use peer to peer networking below some number of participant threshold.

1 more reply

j / k navigate · click thread line to collapse