Can one use the p-value to perform hypothesis testing instead of comparing the test statistic to the critical value at a given significance level?

Question

In hypothesis testing, if a question states the level of significance, which is alpha, then does it necessarily mean that we have to use the classical method? (Means finding critical value and then looking at the table). What if I use the p-value to conclude the hypothesis that it is rejected or acceped. Is it correct?

For example:

A random sample of 64 bags of white cheddar popcorn weighed, on average, 5.23 ounces with a standard deviation of 0.24 ounce. Test the hypothesis that
$\mu= 5.5$ ounces against the alternative hypothesis, $\mu<5.5$ ounces, at the 0.05 level of significance

(This is a question from Probability and statistics for scientist and engineers 9th edition) Look at the example.. (above provided) I have been taught to solve this question by finding the critical value and then looking at the table but what I want to do is to use the p-value and then looking at normal-distribution table to state the hypothesis if it's accepted or rejected.

Answer 1

I am trying to understand hypothesis testing

I'll leave aside meta-commentary about whether hypothesis tests are a good idea or whether the specific example question you supplied is a good candidate for a question about hypothesis testing and respond to the direct question here. That's not to imply I necessarily disagree with the comments, I simply choose in this case to stick to what I see as the framing of the question.

Glossing over some details, the Neyman-Pearson approach is effectively choose some probabilistic model, choose a test statistic, set a significance level, determine a rejection region from those, then examine whether the test statistic calculated on the sample falls into the rejection region. For simple cases, you can conventionally define the critical value as the least extreme value in the rejection region.

There's an equivalence between the Neyman-Pearson approach and use of p values: If all the steps are done correctly, adopting the rejection rule "reject if $p\leq\alpha$" is entirely equivalent in the sense that the two tests should always reject the same cases and not reject the same cases. That is, for any given sample, you should make the same decision in relation to the hypothesis being tested.

It's also entirely reasonable to do this (use a p-value to make the decision) within a Neyman-Pearson paradigm. One can simply define the p-value itself ― which is simply a transformation of the first test statistic ― as the test statistic of interest and literally everything behaves as it should, entirely inside the paradigm. It's not some weird Frankenstein methodology, it's simply a transformation of a test statistic, no weirder than moving from using say $T$ to $|T|$ as a test statistic when performing a two sided t-test.

does it necessarily mean that we have to [...] finding critical value and then looking at the table

As I lay out above, it's already what you're doing when you use a p-value. (But please do it with "open eyes", not blindly; understand what the attainable significance levels are, understand the properties of any approximations being used and so on.)

If define that your test statistic is the p-value, then your critical value can simply be chosen from among available $\alpha$ values, the labour having been undertaken in the step of transformation between initial test statistic and p-value.

[Ggjj's point in comments about permutation tests etc has some relevance here. If you're used to using them the distinction seems pointless]

If both the original test statistic and this transformed one are treated in the same way, then the decisions should be identical in every case (i.e. the tests should be equivalent).

However, if your question about what you have to do relates to what you're expected to do within a subject you're being taught, we cannot address that.

I have been taught to solve this question by finding the critical value and then looking at the table but what I want to do is to use the p-value and then looking at normal-distribution table to state the hypothesis if its accepted or rejected.

If you do it both ways, what happens?

Indeed, try it on dozens of different examples.

If you can find a case that differs, figure out exactly why; that will enlighten you about something you're doing (arguably wrongly).

Answer 2

If by below you mean the $p$ value:

What if I use the p-method to identify the hypothesis. Is it correct?

Then it depends. Typically, people use the Fisher-Neyman-Pearson approach to null hypothesis testing, which includes the set alpha level for your $p$ value. In this specific case, you can only claim statistical significance if the $p$ value is lower than alpha.

However, this is only one paradigm within frequentism. The neo-Fisherian approach completely abandons setting alpha altogether, but that is less common. There are also Bayesian approaches which neglect the absurdity of null effects completely with things like region of practical equivalence (ROPE) testing instead. However, when using the F-N-P method, we usually can only conclude statistical significance if our $p$ value is within the cutoff. So it will depend on what you are doing.

What if I use the p-method to identify the hypothesis.

I'm not sure what is meant here, but I assume you mean whether or not you can assert some hypothesis as being true or false. A low $p$ value only tells us the probability of finding a statistical estimate from the model, given the null hypothesis. When the probability is low enough to fall below our set alpha, we can only reject the null, but we can neither accept the null, nor accept the alternative, based off the $p$ value alone. By the way, alpha and $p$ are not the same thing. Alpha is essentially a cutoff which helps try to prevent Type I error by making the threshold low, where $p$ values below this provide us less chances that this is made (though this hardly actually works in practice).

Answer 3

A random sample of 64 bags of white cheddar popcorn are weighed. On average, they weight 5.23 ounces with a standard deviation of 0.24 ounce. Test the hypothesis that = 5.5 ounces against the alternative hypothesis, <5.5 ounces, at the 0.05 level of significance.

In the above, it's not clear to me what you mean by = 5.5.

If you mean "the true mean of the production line is 5.5", I would assume that bag weights are normally distributed and independent from each other (they are i.i.d). So I would calculate the t-statistics given the sample of n=64, mean=5.23, and sd=0.24, in R:

diff <- 5.23 - 5.5
stderr <- (0.24/sqrt(64))
tstat <- diff/stderr # => -9

How extreme is this t-statistics?

p <- pt(tstat, df=63)

p ~ 3.24e-13 which means that in a world where the factory produces with weights from normal distribution with mean of 5.5, you are extremely unlikely to see a sample of n=64 with mean as low as 5.23 and sd=0.24.

---

If instead you mean "a bag plucked at random from the production line is < 5.5 ounces". I would calculate the z-score given the observed mean, standard deviation and reference weight (5.5):

z <- (5.23 - 5.5)/0.24

Then map this value to a t-distribution with 64-1 degrees of freedom:

pt(z, df=63)
[1] 0.132

This means that given the data and assumptions, ~13% of the bags is expected to weigh 5.5 ounces or more.

---

Either way, I don't see the point in setting a significance threshold, at least in the way the question is phrased.

Answer 4

Comparing a test-statistic to a critical value at a significance level $\alpha$, and computing the associated p-value and comparing it to $\alpha$ are equivalent ways of performing hypothesis testing.

Answer 5

Thanks to the OP for indicating where the question came from (wish more OP's would do so). This book is often used for undergrad stats for engineers (i.e. applied statistics, for non statisticians), and is purely frequentist (except for the very last chapter, which briefly intriduces Bayesian statistics, over 10 pages...). The textbook does not even mention bootstrap, or permutation (a sad situation, typical of many (most?) common textbooks). That specific question is at the end of chapter 10, "One- and Two-Sample Tests of Hypotheses".
Even w/o the background/context of the textbook, this is a classic industrial QC application, where one is checking if the manufacturer is actually "cheating" its customers (a situation which could be extremely costly for the manufacturer, if it were true).

So it is clear that a "classic" frequentist hypothesis test is called for in this case.

Now the OP asks whether the fact that the questions specifies a significance level $\alpha$ implies using the "classic method" (not sure what is meant by that?) or can she use the "p-value". But the p-value only makes sense if we have a significance level, because we can only reject the null if $p<\alpha$. So the mention of $\alpha$ does not at all rule out the use of a p-value.
In fact, the 2 are the same; the "classic method", will compute a t-statistic (in this case, for this test of a single mean), then will look it up in a table (or a computer will do that), and will see that the probability for obtaining this test statistics, or more extreme, is $p$, and will return $p$, the p-value. So the 2 methods are one and the same; yes, you can do the work by hand, and use a table, or you can let the computer/calculator, etc. do it and give you the p-value. It is the same statistic being applied (which I think is what is most important for the OP to understand in this case)
Then, the student will compare $p$ to $\alpha$, and decide to either reject, or fail to reject, the null hypothesis, that the manufacturer gives its customers an average of 5.5oz of popcorn per bag.