I ended up making a big stink elsewhere and they got the repo down. Funny enough, their heads of security told me they'd use my disclosure to push the execs into building a big bounty program. Long story short, their CISO told me on the phone that what I found wasn't a "bug", and that if they did a bug bounty program, they'd go bankrupt.
Ha! Classic Telco. I've seen some in my country and they are an impressive mess of legacy (and many times redundant because of all the M&Âs) applications and undocumented integrations made by an army of low paid outsourced integrators.
Also, low effort mode overall, like your CISO friend there, who probably just wants to survive for sufficient time too jump ship.
His bug bounty speech doesn't hold, as they can start with a very low bounty and increase over time to get the interest of higher skilled people and reach more complex bugs, having total control over spending.
Also, Black hat experts probably have those already mapped and are selling them to the highest bidder, and with privacy regulations getting stricter that "bug bill" will come to them sooner or later.
I believe it. When I worked for them a few years ago, their internal security was pretty bad, and they had tons of random teams with no security guidance, governance, etc. I think the only reason they could operate at all is nobody is trying to hack them. It might be a little bit better now, but knowing the scale and state of things, there's no way they've magically knitted everyone up into properly managed AWS Organizations, to say nothing of actually supporting individual teams' security needs.
The "good news" about your discovery is that it probably was limited to just one tiny system, because everyone maintained completely independent systems and didn't have access to anything else - not because they weren't allowed, but because you didn't even know what other systems there were, much less know how to request access, and virtually nothing internally used SSO. The only way to learn about what other systems there were was to walk around the floors of the Comcast Building and ask random people what they do.
I have no sympathy for these companies. Fix your shit, secure your users. Don't? Well I guess I'm posting these non-bugs all over the net for criminals to enjoy.
I appreciate the honesty! I do think that any company that offers a bug bounty program already has their security in order, to the point where they can no longer find any obvious issues themselves anymore. Not having a bug bounty program implies they don't really trust themselves yet, or haven't reached that level of maturity yet. That probably covers most companies though. I know the codebase I inherited at my current employer wouldn't pass even the most superficial security check. Its only line of defense is that it's only on private networks. Until it isn't. I wonder if I should do a google to look for any public instances... just did, I'm only finding a lot of search engine spam thankfully.
This is why the bad guys win so often. Even companies that do pay bug bounties often pay very little compared to what exploits could be sold for.
No idea if it happened in this case because this is the first I've read about it, but it has always struck me that the websites of campaigners get progressively worse as the battle gets more and more difficult, and fellow campaigners drop out, leaving the diehards with both the campaign and maintaining the overly-ambitious campaign website that someone earnestly set up. There always comes a point where someone just hacks the HTML by hand.
Either way, I wish the guy peace of mind and hope someone can help him with his medical bills.
> Some chip makers have hidden latency and jitter issues from common tests that are in use by consumers and even ISPs. Ping CANNOT be used reliably to test for latency or jitter.
https://web.archive.org/web/20210806220209/https://badmodems...
> The Intel Puma 6 and Puma 7 chipsets have a performance issue in a part of the packet processing that is specific to TCP/IP, which went undetected for awhile because "ping" packets are not TCP/IP. [...]
(And/or because just running "ping" a few times might not catch the issue if it doesn't happen at the same time as the lag spike.)
The badmodems guy seems to be upset that he is outsmarted, honestly. Thanks to him for the good he has done, but if he can't handle being in a room with others smarter than him, this departure is for the better.
He seems to know what he's talking about, but is also invested enough to become a potential sell out.
But the entire thread was hard to follow on that site. There wasn’t much context.
I was given a modem by my ISP that had a Puma 6 chipset and it was a nightmare - it would completely cut out multiple times per day, latency was all over the place, and it was a truly terrible experience.
Doing a bit of digging I was able to short-circuit a lot of troubleshooting and replace the modem, which immediately solved the problem. It truly was negligent how bad those chipsets work - even for basic tasks it was nearly unusable.
Imagine hundreds, or thousands of houses out there all with 25+ year old coax jacks in walls, 4-way splitters jammed into ceilings, leaky outdoor connectors that have been chewed on by squirrels, etc.
It's a miracle that DOCSIS3 works at all.
I'm vaguely familiar with the issue that the axe-grindy guy in the original webpage referenced here is complaining about, specific to some series of chipsets used by a popular cheap cablemodem for a few years of manufacturing run... But what he's found is actually a tip of an iceberg of terrible about why maintaining last-mile copper last wireline infrastructure (POTS wiring/DSL or coax for cablemodems) is a losing bet in the long run.
all the above describes my apartment connection in Astoria. and the neighbors were DIY tapped into it too. At any given time I lived there there was an appointment scheduled or being scheduled to have a field worker try to fix the downtime issues. they never did and i moved out
> Then,, KevinDS on DSLReports was just a incredible ass to me. It was epic. I believe he also lied to me in PM when i tried to get him to help. I was already unhappy with the future that I was looking at. He then just said nasty things. I was done.
Seems to be partially self-inflicted ?
xymox1 wrote :
> You know morons like me should not use Wireshark....
kevinds responded :
> I was thinking more along the lines of,
> Morons like you shouldn't be telling the grown-ups how to secure their networks...
> Sorry, not sorry.. ;)
Now the "grown ups" part is clearly not nice (but then what do you expect when you're starting to lay blame even though you don't know how Internet networking works?), but the "moron" part is clearly mirroring his initial self-deprecating remark, rather than a direct insult.
Also, as cramer says :
> (If anything, he doesn't understand the magnitude of the windmill he's charging.)
----
Wait until he learns how consumer routers typically have the IPv6 firewall off by default (if any), and there's no (need for) NAT (which has never been supposed to be used for security anyway, and also properly set up IPv6 suffixes are too big to be scanned), so "internal/home" devices are "directly" on the Internet !
I truly hope they can feel proud, without any guilt.
"Badmodems.com is worth something, so, with medical expenses in front of me, its time to cash it in"
I think his efforts to publicize the issues with bad modems was worth something to modem users.
But there's nothing to cash in.
You might think "that only moves the attack surface" , you are both right and wrong. Technically it is moved but you eliminate LAN based, wireless (think wpa) and untrusted network devices from the equation and reduce them to one attack surface. Not only that, the threat actors most relevant to most people's threat models are not able to operate or operate with reduced capability to the most part when the attack surface is not a home network/device managed by individuals (and crappy vendors,isps,etc...).
It can pay for itself, don't worry about network device security, just get the cheapest yet fastest stack amd VPN through it.
Whereas securing your own network keeps them intact within your infrastructure.
Cloud is not always the answer. Learning to network and compute, is.
The site seems to be focusing on latency/jitter issues: https://web.archive.org/web/20210506114456fw_/https://badmod...
I'm not entirely sure what to make of the information presented. The page above includes an embedded video demonstrating a difference between ICMP and TCP load-testing: https://www.youtube.com/watch?v=15cJ400yR_E
I'm not knocking that there is interesting data, I'm rather trying to consider potential confounding factors that could call the conclusions being drawn into question.
In particular, I have two questions:
- This doesn't talk about "bufferbloat" (https://en.wikipedia.org/wiki/Bufferbloat) or internal buffering happening inside the modem, only latency/jitter. I'm honestly completely naive about the relationship between the two concepts, but I thought they had reasonable overlap, and could even potentially go some way to explaining the dynamics of what's going on. Why not?
- The video seems to just be TCP- and ICMP-pinging 8.8.8.8. Surely that address is sufficiently hammered that it probably implements fairly aggressive rate control algorithms...?
https://web.archive.org/web/20211002154145/https://www.badmo...
Sometimes you're just done. Makes sense to be clear to everyone involved and put it behind you.
Society progresses, one battle at the time...
Thanks for all the fish!
On the other hand, it seems the issue that was the straw that broke the camel's back is saying that ICMP equals access, and access could mean just ICMP to some (I can access this thing via ICMP), or access could mean full takeover to others (now I know it's there, and therefore it's exploitable to anyone with time & energy).
It's really stupid to get upset about self-proclaimed security experts arguing about stuff where they haven't even clarified what they're arguing about.
Honestly, though, it's a non-issue. Some ISPs use RFC 1918 addresses, and some of those addresses are reachable (via ICMP) from consumer endpoints. So what? That's no less and no more secure that any other set of intermediate hops in ISPs' networks.
https://news.ycombinator.com/item?id=15781474
> Some chip makers have hidden latency and jitter issues from common tests (badmodems.com) 220 points by based2 on Nov 26, 2017 | hide | past | favorite | 42 comments
So I am the Badmodems.com guy. I started getting emails to the badmodems email and I came looking for this thread.
I did fight the good fight. I took out a whole division of Intel. They sold off the connected home div and the Puma. Then the Puma has now pretty much died out.
That last battle tho, with the cable co admin network. OMG. That was some crazy shit. My health is better now and putting all that crap behind me was the right thing to do for my sanity..
My fav part of the whole Intel battle was when i did a 6 hour deposistion in the class action lawsuit. Arris lawyers and Intel battering me for 6 hours. All under oath and videoed. That was SO AWESOME. It was a intense battle. I won all of it. They never pinned me down once. I can also now discuss it and Intel did internal testing and even hired a outside testing lab. They did this really early. The discovery process got emails that confirmed horrendous performance and in email they decided to keep selling it with defects they knew they could not fix.
The cable co admin network thing,, oh gawd what a mess. There was a HUGE comcast network faceplant right when I was working with a guy in europe and with Comcast. Comcast as a whole pretty much fell over in most cities. I never found out what happened. I think Comcast was trying to patch holes and borked their own network.
I am glad to hear people got something from all my efforts. I did save everyone on cableCo systems worldwide from the Puma/Intel. They literally had plans to dominate the world and take over the home while destroying the only competition, Broadcom.I did, pretty much single handed, prevent really nasty devices from polluting the world and now have sent that whole chip into a grave.
The code that makes these devices fail uses up lots of UDP ports which suggests that the issue may be with the NAT implementation. I have done some searching online but haven't got a definitive answer on what exactly the issue is (other than it causes latency and hitter under certain conditions).
I'm slightly sceptical these modems (really modem routers) are as bad as this site previously suggested. It seems like the only use case that breaks them is some specific games where there is a huge amount of latency critical UDP traffic.
http://web.archive.org/web/20211002154145/https://www.badmod...
---
I feel bad for him. Fighting for good security or even a better product shouldn't cost people years of their lives but as long as it doesn't come at the expense of the product's bottom line, there is little reason to compel change as it doesn't impact the bottom line which is what decides whether something is made for most big corporate outputs.
The same argument could be said about "Tech Debt" or non-UL labeled products and a myriad of other causes.
Zero fucks will be paid to anything not socially cool even if it is the responsible thing to do.
That's why we live in the world we live in now.
It doesn't take a rocket scientist to figure out why the world is crap. But for the most part, it takes that "title" just to have a microphone loud enough to move the needle.
This little part of life is what I'm currently struggling with in my post-Covid decision making.
No idea what his most recent issue is but hey, thanks for your work bud, it was useful to many. Time to take a break!
For example:
I was well aware that you needed to get a MAC address whitelisted and then from there a config, including your speed tier would be pushed. I somewhat assumed DHCP reservations were used for that, on a management network, with TFTP for the config file. It makes sense.
This was also solidified by that fact that in most cases during an internet outage, my routers IP would revert back to an RFC1918 address.
Ive never used "landline" services from these providers so SIP wouldn't really be in scope but its not surprising that would be on a separate network and there may be some ability there to "sim-ring" calls. Thats standard in a lot of SIP implementations and probably a feature even a home user would want (ie: ring my cell phone and allow it to connect if phones on the modem dont work).
Things like redirecting traffic etc would get noticed with a ton of stuff being SSL/TLS encrypted. But i would agree some/a lot of this should be using TLS as well.
That said I have never mucked with any of it because.
1. I consider the modem, which i own, to be untrusted and while its inside my DMARC, its outside my security perimeter. Its not directly managed or controlled by me. Anything on it or passing through it should be considered hostile and subject to inspection or filtering.
1a. This is 100% of the reason why i run my own firewall, separately, from a device managed by the ISP and why i avoid AIO style modem/router/firewall devices.
2. I am unsure what mechanisms an ISP may employ to ensure certain things (like upstream/downstream configs) are not tampered with. This could be as simple as a hash check monthly or when billing rolls over to ensure my version is the same as theirs on their servers. Changing that could get be out of bounds with the TOS and canceled, which i have few other options.
So its incumbent of me to basically mind my business. I can also see me "unleashing" my speed tier could impact others on my node, which may cause calls from other customers and an investigation shows that theres a sudden over subscription outside of their norms/standards. Again could be considered malicious and cancel my service/blacklist me or the address.
WITH that said, i do understand the concern with respect to AIO style devices. But again i would consider anything on the ISP network to be "red" and no different from the relative hostility of the greater internet. But i dont see the concern with DHCP traffic and arp traffic, that seems normal, even on a ISP net, its how devices get online and authenticate and find the next hops on the network.
ISP's should do better about segmenting that though, and should probably not provide an AIO solution in general or if they do have one with actual phsyical segementation (ie a box with 2 boards independent of each other connected the same way a modem and router would separately) but I understand why they may want to have simpler setups as well from a customer support standpoint considering the average technical prowess of their userbase.
