Last year they posted about using New Relic, Datadog, and Grafana. Would this ‘silent deletion of log data due to quota’ problem be characteristic of any one of them in particular, or is it something we have to watch out for with all of them?
It implies that you guys are generating the playlists on the fly, tracking the client requests, then feeding that over to your transcoder - which then needs to get the original, seek, and transcode. Why bother?
Two answers.
First, it does save money. A meaningful percentage of videos on the internet are never watched in the first place, and an even larger percentage are watched soon after upload and never watched again. We're able to prune unwatched renditions, and if they happen to be requested years later, they're still playable. Transcoding on the fly lets us save both CPU and storage.
Second, it is ridiculously fast. Our median time-to-publish for a 5-20 minute video is 9 seconds. We had a customer (God bless them) complaining a few months ago that it took us something like 40 seconds to transcode a 40 minute video, which actually was slower than normal for us. If you do an async transcode up front, you're looking at 20 minutes, not <1 minute.
Blog post on this: https://www.mux.com/blog/how-to-transcode-video-100x-faster-...
I worked on something similar a while back, and the data that helped me make a call on whether or not we should transcode on the fly or store renditions was looking at analytics for how often the files are accessed.
I figured out that a large file being transcoded and stored would use more compute resources in ~15 minutes than it was likely to use over the span of _several years_ if it was transcoded on the fly. In a situation where you don't know if the company will exist in several years... You opt for the choice which allows you to stack on the storage later if it's necessary.
That's probably one of very few times I've ever applied YAGNI properly. That was ripe for over engineering
Since it's your own player you can hint to the backend.
Do you dynamically generate the manifests too? Or do things get transcoded on request?
