2. This only looks at successful things. We must examine unsuccessful things to see if there is any real predictive ability.
We don't make such jokes about seismologists though. Why not?
It's rather short of that of course, but after hearing about an outbreak of flu in China on Canadian television in a hotel lobby on holiday from the UK, very early January 2020, I think the 2nd [*] - I heard approximately nothing about except via HN, which included 'this will be bad/global', until March.
[*] (For a while I remembered, because it was notably earlier than UK newspapers kept claiming as a start date, or 'detected in China', in their charts from March. I didn't claim to know the correct date, but obviously that was an upperbound!)
Do you have data to support this? Anecdotally (from own experience and tech workers I know) I feel that remote work has been growing steadily over the past decade, with a an increase in job ads mentioning remote (even if they went out their way to say "office only") but I don't have data either way to back this up. It feels that long-term changes in tech as well as management culture were making remote more common in the long term, and the pandemic accelerated a trend (much like the shift in the movie industry from cinemas to home streaming).
For example, the infamous Dropbox comment: https://news.ycombinator.com/item?id=9224
However, remote work didn't become a trend because of the pandemic, it became the default scenario for many jobs that don't actually require on-site presence but that due to cultural inertia nevertheless often still happened in a colocated office pre-2020.
Remote work has been a trend since long before 2020. With the pandemic the proposition in many cases now simply was "Either work remotely or see your company go out of business.", which had many companies reevaluate their ways rather quickly.
That was not difficult to do, even in Late December. Most discussions were taking it really casually, and believed that none of that would happen, which is exactly why it happened and how I predicted it. People are be filled with unfounded optimism, overestimate their skills, and underestimate their bad lack.
Combining the three with the belief that most humans are incapable of understanding exponential and autoregressive processes, it became clear.
But all of this is anecdotal evidence and could very well be an instance of a bear predicting a depression, after all, bears predicted 10 out of the last 3 economic crises.
I've been 100% WFH in tech for 20 years, and it got easier every year even before covid. What you had to predict, wasn't the pandemic but what tech would be hot enough employers wouldn't care where you worked.
I think it was more of a catalyst.
Edit: for those downvoting, here’s the data:
I am actually surprised as it seemed that with IBM, Yahoo, and various other companies dragging everyone back to the office in 2017-2019 that remote work was in retreat.
Reminds me of punks who'd shop at Hot Topic.
2. There is nothing wrong with “punks” shopping at hot topic, whatever you mean by punk.
If my priority is to know about interesting topics so I can be informed on them when they take off then this is the right methodology. HN might be a bit inefficient in time, but this suggests that new tech trends will generally not arrive without having had some advance warning on HN. This is what I need - as a consultant it's useful to know what the client is talking about when they bring up something cutting-edge, and useful to know about cutting edge things to recommend as relevant.
Do all of these graphs taken into account actual "hype" or just the existence, and number, of posts on hacker news? I would like to see the graphs adjusted for discussion. Take number of comments into account, analysis of sentiment in the discussion, etc. Are they actually being hyped?
For instance, the description above the Bitcoin graph gives a link to the very first mention of Bitcoin[1] but the link goes to a post with no comments.
I guess number of posts and timeline of posts could indicate trend but should we call it "hyping" those trends? I would imagine that would come from the averaged response sentiment of the discussion on those posts.
The variety of tech subjects submitted by hacker newsers is very large, and mostly about the most innovative and financially rewarding domain: tech.
So sure, hacker news does predict every upcoming trend since someone will be posting something about each and every plausibly promising thing.
HN talks about Bitcoin/crypto frequently, but the overwhelming opinion for years has been that it's pointless.
That will certainly be a huge miss, similar to the dismissal of easy file syncing. DeFi/crypto/NFTs seem to be gaining real traction and HN hasn't accepted it.
This is a segment of the population that would be replete with model train enthusiasts, RC plane hobbyists, and HAM radio operators back in the day. So, yeah, things of interest in the tech space would be of even more interest to people on HN.
But yeah, they also picked the "winners", they really have to filter for all topics discussed by HN and then find out how many of those became "future tech" and how many fell by the wayside. Because if it's 1 in 100, it's not a really good predictor.
And if something becomes mainstream, then of course it'll be talked about. That's kind of how it works.
But ultimately what makes something successful is investment in that thing in both time and money. HN is very VC heavy tech crowd and of course it should lead in trends where VC tech crowd creates out of pure effort and money.
Telsa and 3D printing is an interesting example. If it isn't strictly a digital good, HN might not be ahead given the audience.
I don't know if it's a good thing or not, but I've definitely gotten into the habit of opening the comments of a post before the link itself.
Discussions on HN and the articles they comment on can buy into the hype of the day to just as large a degree as anywhere else. Go back a few years. How many people were confidently predicting that widespread door-to-door autonomous taxis were just 2 or 3 years out?
And that is okey.
Hacker News is good about surfacing new technology; Hacker News is bad about figuring out how that technology will be leveraged.
So, you take something like Dropbox. Yeah, it pretty much is just mounted FTP. And the average HN is looking at that and thinking "I could do the same thing with a bash script, what's the big deal".
Only recently the SV/HN community is catching up and waking up to cryptocurrencies - and still hesitantly.
I love reading these sort of things from the past.
The hardest part is keeping an open mind for jumping on early. Hard because 99% turn out to be dud anyway, not to mention our crowd are skeptical bunch.
My biggest "regret" is ignoring Bitcoin/Ether.
Saw Bitcoin posts when it was $5.. nah it's a scam.
Oh it's gone up to $10.. nah it won't go up further.
Went up further.. nah won't buy bcoz already too expensive.
Ether announced.. lolz nobody going to buy another coin since there's already Bitcoin..
Ether goes up.. WTF.
That said, unless you were a True Believer, if you bought bitcoin at $5 you probably wouldn't have any left by now. You likely would have sold at $100. Or $1,000.
I know a lot of people that got in early. But only one made millions because he thought it would be the next big thing. Everyone else jumped ship at some point because they weren't True Believers.
AMD, Square, Cloudflare, Fiverr, Apple, Nvidia, TSM, Twilio, CrowdStrike, Zoom, Atlassian, Moderna, and Shopify.
Maybe a 5 year 500% return compared to the general market's 100%?
I'd say your fat stacks of $ would have you believing it's a pretty good predictor of something.
I guess you could grab a list of every successful and defunct company listed on Crunchbase/AngelList/etc, but there would still be bias, as flopped companies may never have got to the point of creating a profile there, or they scrubbed the profile later.
For example, with this metric, how can we tell the difference between people constantly dismissing Bitcoin or people constantly praising it? Just because you mention it doesn't mean you're mentioning it in a positive light.
(I do this consistently myself.)
That does not mean this is the perfect (or even a good) metric, but I do not think that predicting tech trends is the same thing as embracing them.
> For each topic I counted how many times it had been mentioned in a post title
I suspect that's the problem. A lot of trendy discussion happens in comments; I think that's going to be particularly true for a topic like remote work.
People are going to extol its virtues when it's tangentially related to a submission, but especially pre-pandemic there's only so many stories you could submit about it? 'Company goes full-remote'? 'How we handle remote working at Company'?
It'd be interesting to search comments too I think. hn.algolia.com supports it.
Discussion itself can be a very poor metric, as when there's a great deal of general agreement there's frequently little (or at least less) as when compared with highly contentious topics.
Research methodology can often be restricted by data availability. Finding novel ways to torture^W explore data can be insightful, though raises its own risks (p-hacking, as an example).
The long-term trend is that the world is becoming a better place. Pessimistic people are biased against that fundamental trend that other trends will develop on. Groups of pessimistic people will not be good predictors of the future.
- What is your definition of “long-term”?
- Can a certain industry or geography get worse while the broader world becomes a “better place”?
- Aren’t optimists just lying to themselves in the opposite way as pessimists?
- Is any group of a single type a good predictor of anything?
- Is everyone on HN exactly the same with the exact same outlook?
Source: https://www.vox.com/2014/11/24/7272929/global-poverty-health...
I was ahead of the curve on Bitcoin, VR, and deep learning because I read about them early here, but they weren't the most popular things at the time. HN hates Bitcoin to this day. But I learned about it from HN and decided that HN was wrong about it. When lots of people are wrong about something, I see it as an opportunity.
Deno (by overly invested Node.js users)
Clojure (ClojureScript) (by type-driven programming aficionados)
Scala (by keep-it-simple squad)
C++ (by literally everyone else that does not develop in C++)
Rust ("why don't you rewrite that in Rust?"-jokes squad)
Docker (by the "I still use a simple VM to boot SkyNet"-crowd)
TypeScript (by "don't pollute my JSDoc annotated codebase with types"-crowd)
Kubernetes (by the "I still haven't installed Docker"-crowd)
JVM, CLR, WASM (by the "metal"-people)
Bitcoin (by reasonable people)
But for anyone passing by, crypto assets are like 20 distinct sectors and ever expanding, with many solutions and competing services providers in each sector. HN typically gets stuck on surface level arguments such as limitations in the "currency" concept and the merchant use sector. It also gets reliable stuck on environmental impact, as well as surface level use of NFTs. Solutions providers in oracles, insurance, IOT, yield farming, yield farming aggregators, SaaS products (information, tax), marketplaces and many more entire sectors are almost completely ignored. All alpha bets.
You can lend and borrow on compound.finance, aave. You can play no-loss lotteries like PoolTogether. You can bet on your beliefs and hedge your investments on markets platforms like Polymarket. You can gamble. More on: https://ethereum.org/en/dapps/
Remember, anyone can use all of this without any arbitrary restrictions imposed by governments or companies.
Remote work trended because of the pandemic, for example, but there was constant chatter about remote work anytime the topic of employment came up, and the monthly job match-making posts even post whether the job / worker is open to remote. Functional programming has had low-level chatter from the beginning (given that PG viewed it as one of the reasons for the success of his company), and indeed, functional programming has grown more mainstream. I started exploring Python for my Perl-type tasks because it kept getting good mentions on HN.
I don't think any of the above ever got to the point where it got so many mentions it would hit a strong 100 in his methodology compared to Google trends. I'm also not sure you could identify successful trends early on, but I think you hear chatter on HN long before other places.
* Dolphin emulator progress reports (I love it when they show up)
* Random posts on x86-64 emulation or emulating a gameboy or something like that. Fabrice Bellard is a hero.
* People who post their very first side project ever made either via no-code solutions or some React/Bootstrap type of thing. In many cases, these things are job boards for a specific niche.
* HN praising Apple for standing up to the FBI (or CIA?). HN hating Apple because of recent events with scanning pictures on iCloud and all that.
* All the things the blog post talks about
* Hardcore learning resources on many topics, one that I actually used: http://neuralnetworksanddeeplearning.com/ (tons of books on Calculus as well, I remember a 1900 textbook called Calculus Made Easy)
* The random physics or biological discovery
* And so much more
The way I see it: HN talks about tech (and interesting stuff outside tech), it's bound to cover tech that eventually becomes popular. But will the Dolphin progress reports become as popular as Bitcoin? I don't think so. It'd be interesting times if they would ;-)
That's a long way of saying "no".
You might want to elaborate a little bit instead of posting just a link to wikipedia.
I think they should have also looked at trends that peaked on HN but didn't (yet) peak on google.
TL;DR: No, most people here are sticks in the mud. If you listened to them you’d think that future laptops would still have parallel ports and floppy drives and communicate via IRC.
The HN community is...not so great...at predicting which things are exciting and the industry should adopt. If it were the industry would have settled on a LISP dialect long ago, the web would have adopted good readability standards (based on thousands of meta-complaints on virtually every top rated post on any topic), and mobile app development would still be a growing and profitable business (and Apple would finally be catering to niche HN reader needs).
During the infamous leap second fiasco of Java in 2012 which brought down all JVM servers by saturating the CPU, I saved the day because I was reading about it here on HN while my colleagues were struggling to undestrand why all our environments stopped working.
It took me a few minutes to realize that we were indeed hit by what I just read, look at ten of my colleagues each closely inspecting logs at different ssh sessions, and gleefully give them a precise description of the problem, an approach to verify it and a solution, adding “it’s all over the news guys, you should keep yourself updated”.
The author didn't control for what was on the front page, how many points the article received, or the amount of discussion on the topic. Not to mention the myriad of technologies that are on Hacker News that fizzle.
Less abstractly, productisation is what takes things from obscurity (e.g. HN) to mainstream audiences, so future trends might grow out of stuff that you see here, but the people who can figure out whether certain tech can become a product (would-be predictors) -hopefully- also have the means to turn it into a product themselves.
* Rust
* Covid
For the latter I remember thinking that it was going to become a big deal soon but no one seemed to be paying attention except for those paranoid HNers.
> My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page.
HN also tracks front-page history: news.ycombinator.com/front. To go back to a particular date all you need to do is give the date, i.e. news.ycombinator.com/front?day=2020-08-24. Seems to go back to 2006-10-09.
> Can I convince myself that checking the HN front page multiple times a day is a useful and productive exercise?
That seems to be a different topic entirely and should probably be the headline of the article. Also, it's clearly a bit cheeky, but confirmation bias abounds nonetheless.
HN is not the same as reddit and isn't doomed to the same faults, but as as a forum gets more popular, experts reduce in percentage, they're more reluctant and careful on what and how they comment, the "populism" of the voting system becomes harmful and all the attention creates financial incentives for bad actors.
As for using it as some kind of blackbox betting algorithm for startups and trends, that's just a bad idea. I'm certain that you will get the same graph structure for all things that never became popular.
Obviously any forum focused on leading edge technologies will discuss advancements long before they become a subject of interest in the general populace. But they'll also discuss everything that will fail or never reach the mainstream sphere.
https://en.wikipedia.org/wiki/Survivorship_bias#As_a_general...
This is not really useful data then. A time series with a big spike will "lose" against an identical one without the spike.
Maybe the amount of days when a certain word was in the front page would be a better comparison metric on the HN side?
Blockchain -> Nope
Hacker News is where you to to learn that self-hosted rack-mount servers in your own basement running a LAMP stack provisioned with self-generated bash scripts is the future.
