Definition of p-values: A p-value is a probability that provides a measure of the evidence against the null hypothesis provided by the sample. Smaller p-values indicate more evidence against null hypothesis. Can someone please explain this in simpler terms or in a language easy to understand?
I know there might already be tons of questions around understanding the interpretation of p-values, however I would ask the question in a very limited form and with the use of a specific example:
A manufacturing company fills up can with mean weight of 3 pounds, the level of significance is assumed to be 0.01
H(0) : u >= 3 -- Null hypotheses H(a) : u < 3 -- Alternate hypotheses
We are trying to perform a one tailed test for the case where the population standard deviation is known, so for a sample mean of 2.92 and a standard error of 0.03, we get the z-score as -2.67, giving us the probability (p-value) of 0.0038 or 0.38% that the sample mean would be equal to or lower than 2.92.
Since the probability of getting a sample mean equal or less than 2.92 is 0.38%, which is very small, doesn't it mean that we should accept the null hypotheses? As the chances of getting a mean of 2.92 from a sample is only 0.38%.
Or am I completely missing something here?
Edit - It has been three days now since I tried understanding hypothesis testing and I think I am almost there, I will try to articulate what I have understood so far and then let me know if there are still any gaps in my understanding
p-values measure the likelihood of obtaining the sample mean that we obtained given that the null hypothesis is true. So for the example that I mentioned, the probability of obtaining a sample mean of 2.92 is 0.038 if that population's mean is 3 (as assumed by the null hypothesis).
Now there could be two reasons for obtaining means of 2.92:
- The assumed population mean (i.e., the null hypothesis) is not correct, or
- the population mean is 3 but due to a sampling error / an unlikely sample we got a mean of 2.92.
Now if we select statement 1, we run the chance of making type 1 error and this is where the level of significance comes into play. Using the level of significance we can see if we can reject the null hypothesis or cannot reject null hypothesis.