While nothing that's been said here is inherently wrong per se, a sample YT page load is ~5s to DOMContentLoaded, and without counting the video content, transfers ~7 MiB worth of requests & ~95 requests for me, and visually, the entire page feels like it loads twice. (I thought it was redirecting, but the inspector says nope, that's a single page load.)

… while yeah… a lower bitrate upfront might lower the required bandwidth and thus, latency, to get enough of a buffer to start playback … all the bloat on the page would be a better first port of call.