Much more than a headache, unfortunately: yt-dlp only remuxes the stream as a post-processing step
after all streams have been downloaded. So this would take my nice on-demand stateless solution that just returns redirects to the raw video URLs from YouTube, and turn it into a stateful application that downloads and rehosts the video files on my own server.
To accommodate that, I'd then need to create a configuration layer that determines which feeds to download and when, since I wouldn't necessarily want every video posted on every channel I subscribe to downloaded to my server in advance.
So it turns the whole project into something quite different and much more complex. If there was something that could remux the streams and then pipe the output stream back out over HTTP, that would be more appropriate. Maybe I can work with bindings to FFMpeg to create something that would work here.