Their lead example is recommendation systems, but I can't say that I have received many good suggestions recently.
Spotify and Deezer both suggest the chart hits, regardless of how often I dislike that kind of music.
Amazon keeps recommending me tampons (I'm a guy) ever since I've had a coworking office with a female colleague in 2015.
For all the data that they collect and all the AI that they pay for, these companies get very little revenue to show for it.
There are 1 billion users, each of which spend x$ on the platform. The recommendation system does not need to get 100% accurate (it'd be very hard to get to something that 10% nonaccidental/nonfraud click through rate for example). It needs to be slightly accurate. The difference between 0.5% ctr and 0.6% CTR or conversion is probably 20% increase in revenue, and much more in profits (assuming fixed cost).
But it’s not, it’s a tractable similarity based solution to the question of, which of millions of ads to show, in order to increase CTR.
The biggest difference is motive: a good recommendation is made in good faith for the benefit of the recipient while ads are businesses trying to turn a profit. Maybe that’s why this meme never dies.
People aren't saying recommendations need to be good, but just they need to be better than they are right now. To justify the cost of investment in AI, AI hardware, and AI developers an AI recommendation system needs to be more accurate than the next best recommendation system. The real point here is that no matter how clever you think your AI system is the end user still thinks it's worse than a very simple system based on "people who bought X also bought Y".
Good example!
Not really, they make a shit ton of money from their recommendations and perform extensive A/B testing to keep track of how much money it makes them. Anecdotes aren't data and large tech companies don't spend money for no reason (especially when they can A/B test the impact trivially). Remember that at their scales even a 0.01% increase in revenue is worth $10+ million per year so they don't need to be perfect to make a shit ton of money. There's a reason ML engineers get paid $1+ million and it's not corporate stupidity.
I bought one toilet seat on Amazon and now it thinks I'm a toilet seat collector. No I'm not going to buy another one any time soon, no matter how many color and size/shape/design variations it presents to me.
I really think I would watch more videos if the recommendations -challenged- what I enjoyed, rather than couching me, but I have to assume that someone has run that experiment and ad revenue went down.
I have been an Apple Music user, but I am subscribing to Spotify just for its recommendations.
Just within electronic music, look at how many genres and sub-genres have been categorized by someone simply as a hobby project. Now updated for the web 2.0 era.
Do you have a memory problem? My weekly have always been 30-70% the same recordings of songs I have already listened to at least ten times. And it's always been this way.
You just bought a light bulb? How about a dozen others?
But I've realized that accuracy is not really a problem for them, because a false recommendation at most slightly annoys the user and get glossed over. Among 1000 wrong recommendations, if one works, it's a win (for the company).
Ironically, this suggestion is probably driven by real-world behavior. Whether your bulb fails due to age (and you'll probably need replacements for your other similarly-aged bulbs soon), or an electrical problem caused the burn (and you might need another replacement soon), or you just want to stock up, having your exact bulb model recommended also saves you a little time on having to check it yourself. :P
I think a lot of "wrong" recommendations are actually right for other users, which seems to be just what you're saying -- they annoy one user but result in more sales for others. From the company's perspective, these recommendation systems are working as expected.
On the bright side, that also means in their best interest to improve accuracy (and drive more sales for more users, instead of just annoying them). Hopefully new tools like Nvidia's announcement result in fewer annoying ads.
- frequently click through on Facebook ads because they're often SaaS products I'd be interested in (and have found a bunch of cool tools I use now)
- think my Spotify suggestions are spot-on. I used to use Google Music which had comparable suggestion qualities at the time, but I feel like Spotify has gotten significantly better at suggestions over the past year-ish.
- think YouTube is the shining example of controllable recommendation systems. Looking at my front page right now, I'm interested in all 8 of the videos above the fold, and almost every video below it -- probably a direct result from actively guiding/curating what videos get recommended to me. My "to watch" queue is hundreds of videos long since I almost always add more and more videos until I get the time to sit down and watch a chunk, which usually turns into a positive feedback loop of more good videos getting recommended.
All three of the above recommendation systems make me enjoy the related product more than I would without them, and probably also directly lead to more revenue for the company (sales on FB, keeping my Spotify sub, and seeing more YT ads).
On the other end of the spectrum, posts on FB and Quora are two examples where recommendation systems seem to make products significantly worse, so I guess it's hit or miss depending on whether what you want out of each product aligns with how the recommendation systems are set up.
Youtube is the worst. I have all the time to open videos (and think about it!) in private tab so that my recommendations don't get messed up. And then of course on phone you don't have that luxury so you have to go clean history manually. And now they added two interstitials before you can actually view video in private.
Shining example indeed
That we just remember the bad examples when den recommendation was bad but not when it was good because we didn't make the connection.
I'm asking myself that for quite some time.
Do you use a tracking blocker. That could also be a reason why you get bad recommendations.
Seriously though - When amazon is showing me all these recommendations, they are charging someone to show these to me. That means I am not the customer of those recommendations. They make money when I browse and I wonder if they make more money doing that than when I actually purchase. Meanwhile to the people selling - they pay to advertise with amazon, and they pay (via a %) to sell as well.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it."
-- Upton Sinclair
In other words the reason might be that there is an incentive to suggest you chart hits. The abilities of the recommendation have probably not much to do with it.
Of course they do. Their real customers, record labels, pay handsomely for that service.
The release radar playlist may have more chart hits, but this one is not super smart. If you listen to a popular artist you will get all its new songs and shitty remix forever, including its chart hits.
That is just part of the game and it is called "machine learning". They get to know you better. ;)
Annoying but only part of the game/training.
Most users won't care what hardware their PyTorch model runs on in the cloud. All that matters for them is dollars per training epoch (or cents per inference). This could be a steal for an alternate hardware vendor.
[0] https://web.archive.org/web/20201109023551/https://www.digit...
Compiler bugs a plenty: the compiler can enter infinite loops just trying to compile OpenCL 2.0 code, taking down your program. If you ever come across such a bug, you're now in guess-or-check mode to figure out exactly what grammar you did to bork the OpenCL compiler.
Oh, and the OpenCL compiler is in the device driver. As soon as your customers update to Radeon 19.x.x.whatever, then you have a new OpenCL compiler with new bugs and/or regressions. The entire concept of tying the COMPILER to the device driver is insane. Or you get support tickets along the lines of "I get an infinite loop on Radeon 18.x.x.y drivers", and now you have to have if(deviceDriver == blah) scattered across your code to avoid those situations.
In practice, you end up staying on OpenCL 1.2 which is stable and has fewer bugs... and has functional debugger and profiler. But now you're missing roughly 8-years worth of features that's been added to GPUs over the last decade.
----------
ROCm OpenCL is decent, but that's ROCm. At that point, you might as well be using HIP, since HIP is just a way easier programming language to use.
Ultimately, I think if you're serious about AMD GPU coding, you should move onto ROCm. Either ROCm/OpenCL, or ROCm/HIP.
ROCm is statically compiled: the compiler is Clang/LLVM and completely compiled on your own workstation. If you distribute the executable, it works. Optimization flags work, there's a GDB interface to debug code. Like, you have a reasonable development environment.
So long as your card supports ROCm (admittingly: not many cards are supported, but... AMDPro OpenCL tooling is pretty poor)
Also, no Windows support whatsoever...
A more detailed explanation (ROCm section) https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...
AMD has HIP, which is closer the Cuda but HIP seems less developed.
But I believe the basic problem is AMD doesn't make sufficiently high end GPUs to compete with Nvidia in ML.
Personally, I don't care what happens in the cloud, just what I can buy. I would note Nvidia does have competition in the cloud from Google's TPUs and I assume any large cloud vendor is going to negotiate with Nvidia. While I'd love AMD to be cost-effective for ML somewhere, it seems they aren't 'cause that's not what they're targeting.
Great ideas are much harder come by. And while you can try and buy your way to the best ideas, it doesn't seem to work all that well.
>So there is a back and forth here and all of the other players that we compete with, they are also economic animals. They don't have any magic elixir or magic algorithm that we don't have, right. We all kind of are doing the same thing and we're all providing great service and value for the marketplace. So I know this a little handy way to give an answer but there is an ebbs and flows around competition. The business continues to be very profitable for us on a net basis. On a gross basis it's incredibly profitable but as I said in my remarks, we have paid, put that in quotes, not in our financials we've provided back to our retail customers about $950 million of price improvement this year.
https://seekingalpha.com/article/4386120-virtu-financial-inc...
Top quantitative firms like RenTec don't rely critically on speed or compute power, but rather on high quality data, which they've also said publicly.
over 2 terabytes per second of memory bandwidth."
[0] https://www.nvidia.com/en-us/data-center/dgx-station-a100/
Tensor Processing Units are too specialized: they can't traverse a linked list, they can't traverse trees. They're good at one thing and one thing only: matrix multiplication.
GPUs are still bandwidth-optimized and are good at matrix multiplication (but not as good as tensor units). But GPUs can traverse trees and new data-structures. Ex: BVH trees for raytracing, or linked lists... or whatever else you need. Its a general computer, a weird... terrible latency computer with HUGE bandwidth... but that's still useful in many compute applications.
--------------
Matrix multiplication is the cornerstone of many scientific problems. But you still need software to manipulate the data into the correct "form", so that the matrix multiplication units can then process the data.
Its in this "preprocessing" or "postprocessing" phase where GPUs do best. You can implement bitonic sort for highly-parallel sorting / searching. You can perform GPU-accelerated join networks for SQL. Etc. etc.
And even then, NVidia's A100 have incredibly good matrix multiplication units. So you're really not losing much anyway.
On the other hand, Google has good reason to hold back TPUs from general public, and instead only offer them on Cloud. That also contributes its limited use.
What is the reason? Just competitive advantage for running Google level AI/ML workloads?
There are processors that are designed only for DNN inference but A100 is not one of them.
$US 200K for startling performance this time.
