Large monopolistic tech companies like Apple and Microsoft can afford to ignore this stuff for years because there are few realistic alternatives. But longer term eventually a disruptive competitor comes along who takes product quality and customer service more seriously.
> For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it
| Though only 5.8% of his game's buyers were playing on Linux, they generated over 38% of the bug reports. Not because the Linux platform was buggier, either. Only 3 of the roughly 400 bug reports submitted by Linux users were platform specific
Don't just a bug report by its cover, judge it by its merits. We're all biased to dismiss them and find an excuse to ignore them. But that just leads to bad software.
You could sink an infinite amount of time investigating and find nothing. At some point you have to cut off the time investment when only one person has reported it and no devs have been able to reproduce it.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.
That seems to be the case described in the article. In such a situation, I think it's dishonest to ask the reporter to expend even more effort when you've spent zero. Just close it if you don't want to do it, you don't have to be a jerk to your customers, too, by sending them off on a wild goose chase.
Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
"Please consider cosmic rays hitting the computer, defective ram chips, weird modifications of the system before submitting the bug. Unlesss you explicitly acknowledge that, your bug will be closed automatically in 30 days. Thank you very much"
> this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities.
If you're having trouble reproducing, tag "needs verification" or something else. But closing a ticket isn't triaging, it is sweeping problems under the rug
The developers who keep jumping through hoops are often less technically sophisticated users who click "yes I reproduced it" without actually testing. So you end up with a feedback channel optimized for closing tickets, not for surfacing real problems. High-signal reporters get filtered out, low-signal confirmations pile up.
The Chromium team gets this right by assigning someone to reproduce reports before asking the filer to do anything. More expensive upfront, but the signal quality is far better and the developers who actually know what they're talking about don't give up after the first round of pushback.
Not directed at you of course, just the proverbial “you” from the frustration of a purchaser of software.
Error tracking and tracing make it fairly straight forward to retroactively troubleshoot unreproducible issues.