In hypothesis testing, does a p-value less than alpha always mean you reject NH? I have learned that a p-value that is less than alpha means you reject the null hypothesis. But this seems to only make sense in the case of lower tail tests. What do you do for upper tail tests?
 A: The p-value applies to upper, lower, and double tailed tests. It is the probability that the test statistic would be at least as contradictory to your null hypothesis as you currently observe assuming your null hypothesis is true. So, for upper tail tests, you are comparing $H_o: a = b$ vs. $H_a: a>b$, in this case, the p-value is the probability that the test statistic would be at least as high as you observe, assuming a=b, so you calculate 1 - CDF of the test statistic under the null hypothesis and see if it meets your type I error rate requirement.
A: I'm not entirely sure I understand the question. I think you're asking about whether a test to reject the null hypothesis always requires that a p value be less than your previously-specified confidence level (e.g. p < 0.05). That's what I think you mean by alpha, although I haven't heard it called alpha before.
If I'm interpreting your question correctly, then the answer is yes, rejecting the null hypothesis requires that p be less than alpha. When I explain p-values, I generally don't use the words "null hypothesis" at all, but rather explain p as "the probability that the results you are seeing are purely due to chance/luck." Much, much more loosely/informally and not-strictly-accurately, you can think of p as "the probability that my research hypothesis is true."
So in an upper-tailed test, the value of your test variable (t in the t-test, etc.) might be greater than whatever the value in your lookup table would be, but the value of p that you look up from needs to be less than your chosen value (what you call alpha).
Using p < 0.05 is more tradition than anything else - it essentially means that you'll get fooled by chance 5% of the time. The most important point is to pick your required confidence level BEFORE you run the test. Otherwise you can easily fool yourself.
