>I didn't hear of actual attacks on people in the wild
You never would. It's a passive attack. It's measuring response time to normal operations to discover secrets.
https://mlq.me/download/netspectre.pdf
"Software based side-channel attacks are particularly unsettling since they do not require physical access to the device."