5
$\begingroup$

I have one-sided hypotheses that I want to test with multiple regressions. Does anyone know how to perform a one-sided hypothesis testing? I heard of different positions regarding this matter ("not possible to test one-tailed with regression" and "for one-sided hypotheses just cut the p-value in half"). Or maybe I need another command in with my statistics program (I use R)!? I didn't think this to be a difficult question but couldn't find a consistent and satsfying solution. Thank you in advance for any helping comments.

$\endgroup$

migrated from stackoverflow.com Jan 27 '18 at 18:55

This question came from our site for professional and enthusiast programmers.

  • 2
    $\begingroup$ About what parameter are your hypotheses? The regression coefficients? $\endgroup$ – COOLSerdash Jan 27 '18 at 19:00
8
$\begingroup$

It only requires minimal manual computations to perform one-sided hypothesis testes concerning the regression coefficients $\beta_{i}$.

Two possible one-sided hypotheses are:

\begin{align} \mathrm{H}_{0}&:\beta_i \geq 0 \\ \tag{1} \mathrm{H}_{1}&: \beta_i < 0 \end{align}

or

\begin{align} \mathrm{H}_{0}&:\beta_i \leq 0 \\ \tag{2} \mathrm{H}_{1}&: \beta_i > 0 \end{align}

The $p$-values provided by R are for the two-sided hypotheses and are calculated as $2P(T_{d}\leq -|t|)$ where $T$ is the test statistic (i.e. the regression coefficient divided by its standard error) and $d$ are the residual degrees of freedom.

The corresponding one-sided $p$-values are $P(T_d\leq t)$ and $P(T_d\geq t)$, for the first $(1)$ and second $(2)$ one-sided hypotheses, respectively.

Here is how to calculate the one-sided $p$-values in R:

mod <- lm(Infant.Mortality~., data = swiss)
res <- summary(mod)

# For the two-sided hypotheses

2*pt(-abs(coef(res)[, 3]), mod$df)

(Intercept)   Fertility Agriculture Examination   Education    Catholic 
0.118504431 0.007335715 0.678267621 0.702498865 0.476305225 0.996338704 

# For H1: beta < 0

pt(coef(res)[, 3], mod$df, lower = TRUE)

(Intercept)   Fertility Agriculture Examination   Education    Catholic 
  0.9407478   0.9963321   0.3391338   0.6487506   0.7618474   0.5018306

# For H1: beta > 0

pt(coef(res)[, 3], mod$df, lower = FALSE)

(Intercept)   Fertility Agriculture Examination   Education    Catholic 
0.059252216 0.003667858 0.660866190 0.351249433 0.238152613 0.498169352

mod$df extracts the residual degrees of freedom and coef(res)[, 3] extracts the test statistics.

$\endgroup$
  • 8
    $\begingroup$ Although this solution is correct, I would like to suggest a different approach that is at once simpler and possibly more instructive: just divide all two-sided p-values by 2 and select either them or their complements (subtracting them from 1) depending on the signs of the coefficient estimates. This procedure is so simple it can be done mentally while scanning across the standard output. No computations of the Student t distribution are needed. $\endgroup$ – whuber Jan 27 '18 at 19:49
0
$\begingroup$

I'm not a statistician, so take my answer with a grain of salt but here goes. p-values reported in in linear regression will be obtained using F-test, commonly used for evaluating least-square fitting problems like linear regression). F-test doesn't have two tail version because the distribution of F-value is one-tailed (because F=t^2) to begin with. Read a more detailed answer here

$\endgroup$
  • 5
    $\begingroup$ It depends on which p-values you are referring to. The test for the overall regression, or for any collection of two or more variables, indeed is conducted with an F test. However, the tests of individual coefficients are usually performed with t-tests, which do admit one-tailed alternative hypotheses. This is the point of the comment posted by @COOLSerdash. $\endgroup$ – whuber Jan 27 '18 at 19:06
  • 1
    $\begingroup$ I thought the comment by Matt to that cited blog page was more accurate than the blog text itself. Deviations from a null on either side will increase an F statistic value (or equivalently decrease the p-value) but the "squaring" destroys the information about the direction of the mean deviations. $\endgroup$ – DWin Jan 27 '18 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy