Twitch: *.ttvnw.net
Netflix: *.nflxvideo.net
Hulu: *.hulustream.com
YouTube: *.googlevideo.com
Amazon Prime: *.aiv-cdn.net
Edit: This is by no means the only way to do it, just a potential way to do it.
For mobile ISP's like T-Mobile, they cannot tell that video content is streamed through a VPN protocol like wireguard (but they could add large public providers to the list).
As a good rule of thumb I tell people to run speedtests from both speedtest.net (which is usually whitelisted by every provider) and fast.com (which usually shows up as Netflix video traffic) to see if 'video shaping' rules are in effect.
Oh, wait, they've already been doing that for <insert Gary Oldman> EVERYONE!</insert>
I think its just network analysis or possibly even those caching servers that are run by ISPs which hold onto content network's most heavily used files.
1. during the TLS handshake, the domain name itself might be sent in the clear if the SNI extension is used, and if the SNI extension isn't encrypted[1]
2. if the carrier knows the IP, then they can do a reverse DNS lookup to find the domain name
[1] https://stackoverflow.com/questions/499591/are-https-urls-en...
https://security.stackexchange.com/questions/172212/how-do-m...
This is an interesting 'answer', but how would 'tiered' unlimited plans work if it was handled by the streaming service and not the ISP? Some allow unlimited 480 while the higher priced plans allow unlimited HD.
> Do you have a reference/proof for the first paragraph about agreements?
---
IMHO, I find this hard to believe myself. The team(s) in charge of implementing this functionality likely don't have the requisite high level of lateral coordination necessary to facilitate agreements with all the major streaming services.
Practically speaking, I see this sort of thing as balancing
1) total network saturation/hard capacity limits
2) encouraging user base to use more data up to thresholds in (1)
3) customer experience (wrt data overages)
likely in that order.
This functionality isn't a first-class user-facing "generate piles of money"-center; it describes limitations and conservatism and risk balance. It isn't a "pedal to the floor" sort of subject, but more of a continuous optimization concern.
I don't really see telcos going to all the streaming services and working out agreements given this sort of status quo/sentiment. It's more the sort of thing that is kept internal.
And then of course there's the fact that this would be playing out for every telco relative to every streaming service.
At that point everyone would just throw their hands up in the air and make a global standard for cooperative ratelimiting, and we'd all know about it. For the first few years it would be obscure and you'd have to google the terms a few times to correctly identify it, and then someone would put a 43% quality JPEG screenshot of the documentation on facebook and it would go viral and get linked to 5G causing cancer or whatnot.
It's just IP ratelimiting.
What a great post.
Whether or not they do this, I don't know. But it is technically feasible.
Now, technically, it is probably pretty obvious that what they ought to do is just limit bandwidth or not, regardless of what it is used for. But the rest of the business, and for that matter the rest of the world, conceives of video streaming as something different than "just using bandwidth", so they'll sign contracts about how video streaming will be treated better or worse depending on this or that condition ("Free netflix as long as we can throttle it", "we'll prioritize Disney+ because they paid us and everyone else gets degraded", etc.), so even if it makes no "logical" sense, network hardware will just have to figure out what "video streaming" is to fulfill the contracts whether the engineers like it or not.
With pipelining or some other protocols the signature is going to be a sustained download that runs at a rate less than the maximum possible.
If the servers are being found through DNS that's another tool to figure out what is going on. If not, the carrier is going to want a list of IP blocks associated with video streaming.
What I don't get is customers caring a lot about whether they are getting 480p vs better. The old NTSC video was outright awful, and the difference between that and 480p is greater than that between a clean 480p and 720p, 1080p or even 4k. In the best circumstances the returns are diminishing but if you are viewing on a little phone screen in bright light what's the point?
And do not underestimate the power of carriers. They are the reason why you can not use mobiles on a plane.
The reason you can't [reliably] use mobiles on a plane is the network is designed for terrestrial use with slow handoffs from tower to tower
The signal is shaped to keep it aimed more-or-less downward, and is intended to hand-off from tower to tower at "normal" travel speeds (say, up to ~100mph)
Could handoffs be designed to work at 500kts? Sure
Could towers be designed to aim a notable chunk of signal upwards into the emptiness of sky and space? Sure
But why? The overwhelmingly-typical use case is that of billions of people on the ground moving slowly
The number of airplanes in the air at any given time is minuscule in comparison
These days most streaming providers use some form of adaptive streaming in which client-side logic decides to get bigger or smaller "chunks" of video based on how quickly prior chunks downloaded. A rudimentary solution for a carrier would be to simply implement logic like, "if throughput to device X > someLimit, add a delay in delivering packets to device X". From the client's perspective, getting the bigger (and higher quality) video chunks will take too long, so it will naturally shift to the smaller (and lower quality) chunks.
In T-Mobile's case, this throttling can be avoided by using a tool such as GreenTunnel which can run in Termux (on Android) and works by spliting the SNI portion of the ClientHello into two TCP segments. Their DPI appliances are too dumb to reassemble the fragments and correctly categorize them as going to a streaming service.
The best part about GreenTunnel on Android is that it runs a local HTTP proxy, which you can adb forward to a PC so that you can watch 4k Netflix on your computer using your unlimited T-Mobile plan (this doesn't count as tethering, as the IP packets originate on the phone).
As per the throttling algorithm, most of the times it's a Leaky Bucket variant (https://en.wikipedia.org/wiki/Leaky_bucket). The variant usually allows short bursts of packets, then throttles down the downstream traffic for long connections to match the configured rate.
A trick to know if the operator is using DPI to extract the SNI in HTTPS/encrypted traffic: play a YouTube video, then do a speed test (e.g. iperf) while it is playing. Two things could happen: either both apps are throttled (no DPI) or only Youtube is (there's some level of DPI).
1. try and see if accessing youtube through a vpn improves the bandwidth (in that case, your ISP is probably looking at both dns requests and connection endpoint ownership)
2. preload and buffer whole videos (es: https://www.technorms.com/35122/preload-buffer-entire-youtub...) this aims to get your traffic usage pattern not having the "usual" shape of a typical youtube session (that is: brief burst of full-speed downloads)
Possible otheroptions:
- agreements with the service providers to throttle users from certain netblocks (the carriers partner with the service providers to some extent e.g. to deploy CDN nodes, so such agreements would be plausible)
- throttling bandwidth (potentially selectively to/from streaming providers) and letting the service figure it out
- separate host names for high res content that can be DNS-blocked
My provider limits me to ~1.5Mbps, but the second I connect to a VPN (WireGuard - hosted on my homes 1Gbps/1Gbps connection), it goes up to ~50Mbps.
1) What does the feature look like, screenshot-wise?
2) Can you confirm the HD/4K option actually disappears or is disabled, or if the site(s) in question just trend toward autoselecting 480p/720p over time?
Like most other comments here I suspect IP-based bandwidth limiting. Given the unbounded complexity scale of keeping the internet actually working :) I can totally see infrastructure being able to single out the activity of a single connection and track what it's doing over time. The chances are the implementation is eyebrow-raisingly impressive but still compact and approachable at the end of the day.
What I can add is that it's to do with IP or DNS monitoring from the carrier or server. My SO who uses Verizon noticed it, and so I setup a home VPN for him, and when connected to it, all throttling disappears and everything is accessible at full 5G speeds. We have gigabit internet, so it's trivial for our network to handle the VPN traffic of streaming.
As soon as I enable a VPN (I use PIA), speeds go back to normal.
