This is probably a simple question but I was watching an online video about a scientific study. For the study, they mentioned that they used a two-tailed Type 1 error rate of 0.05. I know that a Type 1 error is the probability of rejecting the null hypothesis when it is actually true. However, I am not sure what is meant by a two-tailed Type 1 error rate. Does the 'two-tailed' refer to the fact that the hypothesis test used is two-sided? Any insights are appreciated.
$\begingroup$
$\endgroup$
1
-
4$\begingroup$ Yes, it means that the test is two-tailed. The particular phrasing has to do with allocating $0.05$ probability between the two tales, rather than putting all of that in just one tail. $\endgroup$– DaveOct 10, 2021 at 4:29
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Examples of two-sided and one-sided t tests
Suppose you have data as follows: I used R to take a sample of size $n = 30$ from a normal distribution population with mean $\mu = 100.$ However, I happened to get a sample with mean $\bar X - 95.55,$ somewhat smaller than $\mu = 100.$
set.seed(1234)
x = rnorm(30, 100, 15)
summary(x)
Min. 1st Qu. Median Mean 3rd Qu. Max.
64.81 86.85 92.49 95.55 103.62 136.24
I will do a t test of the null hypothesis $H_O: \mu =100$ against the two-sided alternative $H_a:\mu \ne 100.$ I want to test at the 5% level of significance.
t.test(x, mu = 100)
One Sample t-test
data: x
t = -1.798, df = 29, p-value = 0.08259
alternative hypothesis:
true mean is not equal to 100
95 percent confidence interval:
90.49593 100.61132
sample estimates:
mean of x
95.55363
The P-value is $0.08259 > 0.05 = 5\%,$ so I do not reject at the 5% level. Even though I happened to get a sample that is a bit strange, its sample mean 95.55 is not enough different from $\mu_0 = 100$ to reject the null hypothesis.
The P-value is the probability under the density curve of Student's t distribution with 29 degrees of freedom of getting a T statistic farther from $0$ than the observed $-1.798.$ In the figure below that amounts to the sum of the areas in the two tails outside the vertical red lines.
However, if I decide (perhaps after seeing the small value of $\bar X)$ to do a left-sided test of $H_0:\mu=100$ against $H_a:\mu < 100,$ also at the 5% level. Then I get P-value $0.0413 < 0.05 = 5\%,$ so I reject the null hypothesis. In the figure, this is the area in the left tail (only) to the left of the solid red line. (The P-value of the one-tailed test is half of the P-value of the two-sided test.)
t.test(x, mu=100, alte="less")
One Sample t-test
data: x
t = -1.798, df = 29, p-value = 0.0413
alternative hypothesis:
true mean is less than 100
95 percent confidence interval:
-Inf 99.75543
sample estimates:
mean of x
95.55363
Even though both the two-sided and the one-sided test were at the same 5% level, we failed to reject $H_0$ for the two-sided test and rejected for the one-tailed test. This is because the criteria for rejection are different for the two tests. So, in a practical application, it is important to decide from the start (preferably before data are available) whether you need to do a two-sided or a one-sided test.
Notes: (1) In case you're familiar with R and want the R code for the figure, it is shown below.
hdr = "Density of T(29)"
curve(dt(x, 20), -3.5, 3.5, lwd=2,
ylab="PDF", xlab="t", main=hdr)
abline(h=0, col="green2")
abline(v=0, col="green2")
abline(v=-1.798, col="red")
abline(v=1.798, col="red", lty="dotted")
In case you want to see how P-values can be obtained in R, here they are:
1 - diff(pt(c(-1.798, 1.798), 29))
[1] 0.08259619 # 2-sided P-value
pt(-1.798, 29)
[1] 0.04129809
