I'm new to hypothesis testing in statistics.
Say if we were to run an experimental trial on two control groups (split of 50% each), and after a large enough sample size, conclude by looking at just "mean" success rate in both trials that one group was better than the other.
This method seems very intuitive (since we've allowed the mean to settle over a long time), and seems accurate.
However, statistical formuale usually include a standard deviation component when computing the "p" value - (x - u)/(σ / sqrt(n)). Are there cases where would it be wrong to conclude the result of an experiment by just looking at the mean (due to the fact that we neglected the σ factor which would reverse the conclusion) ?