How do you know your ad even worked and you didn't just correlate performance of your ad campaign with some confounding factor(s) you will never be able to account for? you aren't testing for a mechanism here, just whether certain other metrics changed favorably over the campaign period and you make assumptions about the cause. you have no idea if the ad converted or not. how may times are you repeating your ab testing and with what sample size? are you running simulations too so you understand what a successful campaign would even look like based on your sample size? are you underpowered or underdetermined? probably only the most careful advertisers are thinking about these things, and I really wonder how significant the results look after doing these sorts of things.
