Running a software rollout is frequently done slowly, datacenter by datacenter, and during that time some people might see one version and others might see another.
From the users perspective it looks the same as an A/B test, but the difference is nobody was looking at the results...
For example the buy now vs add to cart only on the Amazon one might have been looking at more than just how many products are sold, they might were also been trying to see if they can say reduce impulse buys that result in returns without lowering purchases that do not, in fact the reason they've kept the buy now might be because it actually reduced the return rate as people interacted less with the site and didn't buy additional items that they returned later.
How are these leaked? Did someone hack into something?
It could also be looking for possible A/B flags in cookies/localStorage.
I am all for A/B testing, but the devil is in the details. You can get more users tapping the purchase by moving the purchase button where users are more prone to accidentally tap the purchase button. That doesn’t mean you get more purchases, or that the move was a positive change.
