2
$\begingroup$

I have multiple regression with, say 3 independent variables: $Y=B_0+B_1x_1+B_2x_2+B_3x_3$ I would like to test if $B_2+3B_3$ is significantly different from zero, i.e. $$H_0: B_2+3B_3=0$$ $$H_1: B_2+3B_3\neq 0$$ Can you please help to find appropriate way to test for significance of linear functions of two coefficients as in above example. Many thanks in advance.

$\endgroup$
  • $\begingroup$ What is the assumed distribution of your errors? If they are assumed to be normal, your estimated coefficients will be normally distributed with variance-covariance structure that you can find in any regression textbook. Then you can find the distribution of a linear combination of normal random variables. $\endgroup$ – caburke Jan 7 '13 at 5:30
  • $\begingroup$ Thanks a lot for your response, errors are distributed normally, but independent variables are not fully independent. I want to find method to combine t-tests for Beta coefficients. since t-distributions(normal distribution too) for two Betas can be dependent I am asking is there method to add two t-distribution or if it possible to do this by avoiding summing of two t-distribution $\endgroup$ – Fazil Jan 7 '13 at 5:41
  • 1
    $\begingroup$ This is a standard topic in any linear regression textbook. The statistic for testing the hypothesis is called Wald statistic. $\endgroup$ – mpiktas Jan 7 '13 at 7:37
3
$\begingroup$

If your errors are normal and regressors are non-random, the OLS estimates of the coefficients are normal:

$$\hat\beta-\beta\sim N(0,\sigma^2(X'X)^{-1})$$

Hence any linear combination is normal too:

$$R\hat(\beta-\beta)\sim N(0, R\sigma^2(X'X)^{-1}R')$$

You want to test that $R\beta=r$, with $R$ being $[0,0,1,3]$ and $r=0$. The Wald statistic for testing the null hypothesis $R\beta=r$ is

$$(R\hat\beta-r)(R\sigma^2(X'X)^{-1}R')^{-1}(R\hat\beta-r)\sim \chi^2_q,$$

where $q$ is the rank of $R$, which in your case is simply 1. You have unknown $\sigma^2$, simply plug in the consistent estimate and you are good to go.

This statistic is implemented in practically all the statistical packages which estimate linear regression. In R you need to use function linearHypothesis from package car.

$\endgroup$
  • $\begingroup$ Hi everybody, many many thanks for valuable comments and help. I think Wald statistics you explained above is what I needed. $\endgroup$ – Fazil Jan 7 '13 at 16:49
  • $\begingroup$ Hi @Fazil - if mpiktas has answered your question well enough for you can you "accept" it by clicking on the tick mark next to the question, so it is marked in the Cross-Validated database as resolved. $\endgroup$ – Peter Ellis Jan 7 '13 at 18:27
  • $\begingroup$ Hi Peter, all comments were very helpful and resolved my problem. I'm trying to accept solutions but could not find tick mark. Next to my question there are share, edit, delete and flag buttons and keywords button. $\endgroup$ – Fazil Jan 7 '13 at 19:40
2
$\begingroup$

Your intuition is correct that you cannot just simply add together the estimates of the two parameters. Luckily as @caburke suggests in his comment this is a very standard application of regression and there is a way to do this. The key words to search for are linear combination of estimates from linear regression or (mysteriously) "contrasts".

Given your assumptions, your linear combination of estimates will itself have a t distribution, with standard error equal to

$ s\sqrt{b^t(X^tX)^{-1}b}$

Where b is the vector indicating your linear combination of coefficients you are interested in (in your case, [0,0,1,3]); X is your original matrix of explanatory data (including a column of 1s for the intercept) and $s^2$ is the estimated residual variance.

Most stats software will have a way of doing all of this linear algebra for you.

There are doubtless packages in R (eg the 'contrast' package) that have this conveniently wrapped up if you don't want to do it by hand. A nice little basic function that does it in R is available here: https://notendur.hi.is/thor/TLH2010/Fyrirlestrar/Kafli4/lincomRv8.R. Sorry, I can't identify the author of it, but for the record (in case the link goes down) here is the code:

# A function to estimate a linear combination of parameters from a linear model along
# with the standard error of such a combination.
# lm.result (or model.result) is the result from lm or glm. 
# contrast.est is the estimate.
# contrast.se is the standard error.

lincom <- function(model.result,contrast.vector,alpha=0.05) 
{
beta.coef <- coef(model.result)[1:length(contrast.vector)]
dispersion.param <- summary.lm(model.result)$sigma
    beta.cov <-  dispersion.param^2*summary(model.result)$cov.unscaled[1:length(contrast.vector),1:length(contrast.vector)]
df.error <- summary(model.result)$df[2]
contrast.est <- c(t(contrast.vector) %*% beta.coef)
contrast.se <- sqrt(c(t(contrast.vector) %*% beta.cov %*% contrast.vector))
tvalue <- contrast.est/contrast.se
lowerb <- contrast.est -  qt(1-alpha/2,df.error) * contrast.se
upperb <- contrast.est +  qt(1-alpha/2,df.error) * contrast.se
pvalue <- 2*(1-pt(abs(tvalue),df.error))
return(list(contrast.est=contrast.est,contrast.se=contrast.se,lower95CI=lowerb,upper95CI=upperb,tvalue=tvalue,pvalue=pvalue))
}
$\endgroup$
  • $\begingroup$ For testing linear hypotheses there is an R function linearHypothesis from the the car package. $\endgroup$ – mpiktas Jan 7 '13 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.