I have one-sided hypotheses that I want to test with multiple regressions. Does anyone know how to perform a one-sided hypothesis testing? I heard of different positions regarding this matter ("not possible to test one-tailed with regression" and "for one-sided hypotheses just cut the p-value in half"). Or maybe I need another command in with my statistics program (I use R)!? I didn't think this to be a difficult question but couldn't find a consistent and satsfying solution. Thank you in advance for any helping comments.
2 Answers
$\begingroup$
$\endgroup$
1
It only requires minimal manual computations to perform one-sided hypothesis testes concerning the regression coefficients $\beta_{i}$.
Two possible one-sided hypotheses are:
\begin{align} \mathrm{H}_{0}&:\beta_i \geq 0 \\ \tag{1} \mathrm{H}_{1}&: \beta_i < 0 \end{align}
or
\begin{align} \mathrm{H}_{0}&:\beta_i \leq 0 \\ \tag{2} \mathrm{H}_{1}&: \beta_i > 0 \end{align}
The $p$-values provided by R are for the two-sided hypotheses and are calculated as $2P(T_{d}\leq -|t|)$ where $T$ is the test statistic (i.e. the regression coefficient divided by its standard error) and $d$ are the residual degrees of freedom.
The corresponding one-sided $p$-values are $P(T_d\leq t)$ and $P(T_d\geq t)$, for the first $(1)$ and second $(2)$ one-sided hypotheses, respectively.
Here is how to calculate the one-sided $p$-values in R:
mod <- lm(Infant.Mortality~., data = swiss)
res <- summary(mod)
# For the two-sided hypotheses
2*pt(-abs(coef(res)[, 3]), mod$df)
(Intercept) Fertility Agriculture Examination Education Catholic
0.118504431 0.007335715 0.678267621 0.702498865 0.476305225 0.996338704
# For H1: beta < 0
pt(coef(res)[, 3], mod$df, lower = TRUE)
(Intercept) Fertility Agriculture Examination Education Catholic
0.9407478 0.9963321 0.3391338 0.6487506 0.7618474 0.5018306
# For H1: beta > 0
pt(coef(res)[, 3], mod$df, lower = FALSE)
(Intercept) Fertility Agriculture Examination Education Catholic
0.059252216 0.003667858 0.660866190 0.351249433 0.238152613 0.498169352
mod$df extracts the residual degrees of freedom and coef(res)[, 3] extracts the test statistics.
answered Jan 27, 2018 at 19:33
-
16$\begingroup$ Although this solution is correct, I would like to suggest a different approach that is at once simpler and possibly more instructive: just divide all two-sided p-values by 2 and select either them or their complements (subtracting them from 1) depending on the signs of the coefficient estimates. This procedure is so simple it can be done mentally while scanning across the standard output. No computations of the Student t distribution are needed. $\endgroup$– whuber ♦Commented Jan 27, 2018 at 19:49
$\begingroup$
$\endgroup$
2
I'm not a statistician, so take my answer with a grain of salt but here goes. p-values reported in in linear regression will be obtained using F-test, commonly used for evaluating least-square fitting problems like linear regression). F-test doesn't have two tail version because the distribution of F-value is one-tailed (because F=t^2) to begin with. Read a more detailed answer here
-
5$\begingroup$ It depends on which p-values you are referring to. The test for the overall regression, or for any collection of two or more variables, indeed is conducted with an F test. However, the tests of individual coefficients are usually performed with t-tests, which do admit one-tailed alternative hypotheses. This is the point of the comment posted by @COOLSerdash. $\endgroup$– whuber ♦Commented Jan 27, 2018 at 19:06
-
4$\begingroup$ I thought the comment by Matt to that cited blog page was more accurate than the blog text itself. Deviations from a null on either side will increase an F statistic value (or equivalently decrease the p-value) but the "squaring" destroys the information about the direction of the mean deviations. $\endgroup$– DWinCommented Jan 27, 2018 at 19:48