# One-tailed F-test with one restriction

I have a regression, e.g. $Y=a+b_1X_1+b_2X_2+c_1Z_1+c_2Z_2+e$.

Is it possible test the one-tailed null $b_1+b_2\leqslant c_1+c_2$ against the alternative $b_1+b_2>c_1+c_2$? Notice that this is an $F$-test with a single restriction.

• – Richard Hardy Mar 11 '17 at 15:46

Yes, this is possible.

The general multiple linear regression model is

$$y = X\beta + \varepsilon$$

with the $k$ regressor values, along with a constant vector, forming the columns of $X$ and the $p=k+1$ coefficients arranged correspondingly in the vector $\beta$. When there is no collinearity among those columns, the least-squares solution is

$$\hat \beta = (X^\prime X)^{-1}X^\prime y.$$

It fits (or "predicts") the expectations $E[y|X]$ with

$$\hat y = X\hat\beta = X(X^\prime X)^{-1}X^\prime y.$$

Assuming the "errors" $\varepsilon$ are uncorrelated and have common variance $\sigma^2$, the unbiased estimator of $\sigma^2$ is

$$\hat\sigma^2 = \frac{||y - \hat y||^2}{n-p}.$$

(This is the mean squared residual $||y - \hat y||^2/n$ adjusted by the factor $n/(n-p)$ to make the estimator unbiased.) The covariance matrix of the estimates $\hat\beta$ can then be estimated as

$$\operatorname{Cov}(\hat\beta) \approx \hat\sigma^2 (X^\prime X)^{-1}.$$

Any linear combination $c^\prime \beta$ will be estimated with the same linear combination of parameter estimates, $c^\prime \hat\beta$. To compare $c^\prime\beta$ to some predetermined constant $c_0$ (often $0$), compute the variance of the estimate as

$$\operatorname{se}(c^\prime\hat\beta) = \operatorname{Var}(c^\prime\hat\beta) = c^\prime \operatorname{Cov}(\hat\beta)c = \hat\sigma^2 c^\prime(X^\prime X)^{-1}c$$

and conduct a t-test (either one- or two-sided) based on the t statistic

$$t = \frac{c^\prime\hat\beta - c_0}{\sqrt{\operatorname{se}(c^\prime\hat\beta) }}.$$

For $n$ observations, it has $n-p$ degrees of freedom.

The following R code shows how to perform these calculations and runs a (fast) simulation demonstrating that one-sided p-values do indeed have a uniform distribution when the null hypothesis holds. The required information is extracted from an lm fit using coef to obtain $\hat \beta$ and vcov for $\hat\sigma^2 (X^\prime X)^{-1}$.

To check the simulation, it conducts a $\chi^2$ test of uniformity; the output for this particular set of 1000 iterations is a p-value of $0.4428$, evidence that the code is performing as claimed.

beta <- c(1,4,2,3)       # The coefficients
n <- 20                  # The number of observations
combo <- c(0,1,1,-1,-1)  # The contrast (starting with an intercept coefficient)
sigma <- 2               # The error SD
n.sim <- 1e3             # Number of iterations of the simulation
#
# Prepare to simulate.
#
set.seed(17)
k <- length(beta)
combo.name <- paste0("Contrast(", paste(combo, collapse=","), ")")
#
# Simulate with many different sets of regressors and responses, but all
# using the same beta.
#
p.values <- replicate(n.sim, {
#
# Create regressors.
#
x <- matrix(rnorm(n*k), n)
colnames(x) <- paste0("X", 1:k)
y <- x %*% beta + rnorm(n, 0, sigma)
#
# Test the combination.
#
fit <- lm(y ~ ., as.data.frame(x))
beta.hat <- crossprod(coef(fit), combo)
beta.hat.se <- sqrt(combo %*% vcov(fit) %*% combo)
t.stat <- beta.hat / beta.hat.se
p <- pt(t.stat, n-k-1) # A one-sided p-value
#
# When running this "for real," uncomment the following lines to
# see the results.
#
#   stats <- rbind(coef(summary(fit)), c(beta.hat, beta.hat.se, t.stat, p))
#   rownames(stats)[k+2] <- combo.name
#   print(stats, digits=3)
#
# Return the p-value for the (two-sided) t-test
#
p
})
#
# Display the simulation results.
#
n.bins <- floor(n.sim / 100)
(p.dist <- chisq.test(table(floor(n.bins*p.values)))$p.value) #$
hist(p.values, freq=FALSE, breaks=n.bins,
sub=paste("p (uniform) =", format(p.dist, digits=3)))
abline(h = 1, col="Gray", lwd=2, lty=3)


Yes this is indeed very possible, in fact nothing changes. Use the standard F-test provided by whatever software you are using.

Actually, when you only have one parameter the F-test is just the square of a the same t-test. That is, $t^2_{n-k-1} \sim F_{1,n-k-1}$. As Whuber points out in the comments, you have to be carefull in thinking about 1 and 2 sided alternatives. And in this case, (for the above results to be true) you need a two sided alternative. But this should always be the case, unless you have a really good argument for why the statistic cannot be negative (or positive).

• So I can just compute the F-stat, take the square root, and see if it is larger than my one-sided critical t-value, e.g. 1.28 for the 10% significance level? – user93929 Nov 6 '15 at 14:01
• You seem to be claiming that the one-tailed and two-tailed tests are the same, but obviously they are not. – whuber Nov 6 '15 at 14:04
• @whuber, yes indeed. Have updated! – Repmat Nov 6 '15 at 14:29
• Is it that simple? It's not obvious to me that this should work... – Richard Hardy Nov 6 '15 at 14:38
• So, I can only test a two-sided alternative and not a one-sided alternative with the F-test with one restriction? – user93929 Nov 6 '15 at 15:03