I'm having trouble making the full connection between AB Testing and Hypothesis testing.
Hypothesis Testing Conducting a test to compare two means, we need the standard deviation of both samples, and the two means. H0: Mean of sample one equals mean of sample two.
My understanding of the intuition is we are testing the odds that the results are just coincidence (assuming H0 is true). Therefore the more extreme results the more likely we are to reject the null.
AB Testing Here we are asked for the Minimum Detectable Effect, Baseline Conversion Rate, significance level and statistical power. I think MDE is setting up a confidence interval but I don't really understand.
My question is, how do these two tests relate to each other? Is an AB Test truly just a test of means under the hood?