2
$\begingroup$

Lets say we have the following null and alternative hypothesis:

$$H_0: \mu = \mu_0, \; H_A:\mu > \mu_0 $$

Thus, the p-value in this situation represents the probability of obtaining test results equal to or greater than the results actually observed (under the assumption that the null hypothesis is correct): $$P(t_{df = n-1} \geq t_{obs})$$

Finally, here is a visualization of the theoretical sampling distributions of the two hypotheses:

enter image description here

The red curve and green curves represents the sampling distributions of the null and alternative hypotheses, respectively. The blue vertical line represents the value of the observed results.

Now it is clear that the p-value for these observe results would be very close to 1. But I am struggling to logically explain the conclusion from this result (p-value) in words. Here is my horribly clunky attempt:

"Since the p-value of this t-test is so high, it means it is very probably that any future observations would have higher values than the value of the current observation. And since the mean of the null hypothesis is closer to the current observation than the mean of the alternative hypothesis, it is more likely the current observation came from the null hypothesis than the alternative hypothesis. Moreover, we cannot "accept" the null hypothesis here since it is quite clear the observation does not belong to the null hypothesis. Therefore, we simply just do not reject the null.

My first and second sentence in particular don't seem logically connected. In other words, how can I explain that this high p value "favors" the null over the alternative?

$\endgroup$

2 Answers 2

1
$\begingroup$

You do not give data to go with your question, so I will illustrate in terms of fictitious data, sampled in R, that may be suitable for illustration.

set.seed(2022)
x = rnorm(100, 50, 10)
summary(x);  length(x);  sd(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  20.99   45.30   51.72   51.39   58.66   78.87 
[1] 100         # sample size
[1] 10.20093    # sample standard deviation.

Here is a stripchart that shows the 100 observations.

stripchart(x, pch="|")

enter image description here

If you want to test $H_0: \mu \le 60$ against $H_a: \mu > 60.$ The corresponding test is a right-sided one-sample t test. The test statistic is $T = \frac{\bar X = \mu_0}{S/\sqrt{n}} = \frac{51.387 - 60}{10.20/\sqrt{100}} = -8.4429.$

From a printed table of Student's t distributions, you can find (at least approximately) that the critical value of the test is $c = 1.6604.$ You would look on row DF=99 of the table. if there is one, otherwise find the nearest. This is the value that cuts 5% of the area from the upper tail of Student's t distribution with 99 degrees of freedom. [The exact value below is from R.]

qt(.95, 99)
[1] 1.660391

This means that you would reject $H_0$ in favor of $H_q$ if $T \ge c = 1.6604.$ However, in fact, the statistic $T = -8.4429$ from your data is (much) too small to reject $H_0.$

In R, the test looks like this:

t.test(x, mu = 60, alt="greater")

        One Sample t-test

data:  x
t = -8.4429, df = 99, p-value = 1
alternative hypothesis: 
 true mean is greater than 60
95 percent confidence interval:
  49.69371      Inf
sample estimates:
mean of x 
 51.38746 

It seems that the alternative hypothesis was set up expecting an alternative population mean $\mu_a$ that may be greater than $60;$ instead the population mean appears to be smaller.

The P-value is the probability under the density curve for Student's t distribution with 99 degrees of freedom that lies above $-8.4429.$ This probability is very nearly $1,$ the P-value of the test.

The graph you pasted into your question seems to be for a normal test (z test) instead of a t test [and with a specific alternative (green) in mind]. The plot below shows the density function of $\mathsf{T}(99)$ in maroon, the observed value of the $T$ statistic (heavy blue vertical line), and the critical value $c = 1.6604$ (vertical dotted line).

enter image description here

R code for figure:

hdr="Density of T(99)"
curve(dt(x, 99), -9, 5, col="maroon", lwd=2, main=hdr)
 abline(v = -8.4429, col="blue", lwd=2)
 abline(v = 1.6604, col="darkgreen", lwd=2, lty="dotted")
 abline(v=0, col="green2")
 abline(h=0, col="green2")
$\endgroup$
2
  • 1
    $\begingroup$ Thank you so much for the answer! Does that mean we do not reject the null because the high p-value suggests the alternative hypothesis would have an even higher p-value $\endgroup$
    – Nova
    Feb 27 at 19:33
  • $\begingroup$ Reject $H_0$ because P-value not < 5%. But note that a (possibly more appropriate) two-sided test of $H_0: \mu=60$ vs $H_a: \mu \ne 60$ would give very small P-val because true population mean is far below 60, as estimated by $\bar X = 51.39.$ // Whether you compute the test by hand or by computer, when you're just getting started, it's easy to do a "wrong-sided" test my mistake. My guess is that the point of this problem may have been to let you see what happens when you get the "sidedness" wrong. $\endgroup$
    – BruceET
    Feb 27 at 20:39
0
$\begingroup$

I'm aware I'm entering a minefield here since the correct explanation of p-values in plain English is tricky. Here's my take on explaining the result of a large p-value:

A large p-value indicates that the observed difference, or more precisely the observed value of the test statistics, is compatible with the assumptions of the null hypothesis.

Here compatible means just that and it shouldn't be confused with "supports the null" and it doesn't inform on the plausibility of one hypothesis or the other.

It may be worth keeping in mind that the p-value is a random variable (i.e. it depends on the sampled data) and if the null hypothesis holds the p-values follow a uniform distribution between 0 and 1. In other words, if $H_0$ holds large p-values are as likely to appear as small ones. I think this is counter-intuitive and easily overlooked.

Here's a simulated example to illustrate the behavior under the null:

p.values <- rep(NA, 10000) 
for(i in 1:length(p.values)) {
    x <- rnorm(n= 20, mean= 0, sd= 1)
    y <- rnorm(n= 20, mean= 0, sd= 1)
    res <- t.test(x, y)
    p.values[i] <- res$p.value
}
hist(p.values, breaks= 30)

enter image description here

$\endgroup$
1
  • 1
    $\begingroup$ I grant that "large" P-values are common under $H_0,$ as you show. However, in practice it would be misleading to give the impression that a P-value ≈1 is common. Often, such a P-value indicates that something is wrong: Nonrandom sample, sample from wrong population, wrong test, incorrect use of software, etc. Here the difficulty is that we are looking in the "wrong" tail for the alternative value. // To put it another way, under $H_0$ P-val is below $0.05$ only 5% of the time; also, P-val is above $0.95$ only 5% of the time. $\endgroup$
    – BruceET
    Feb 27 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.