One-tailed F-test with one restriction

Question

I have a regression, e.g. $Y=a+b_1X_1+b_2X_2+c_1Z_1+c_2Z_2+e$.

Is it possible test the one-tailed null $b_1+b_2\leqslant c_1+c_2$ against the alternative $b_1+b_2>c_1+c_2$? Notice that this is an $F$-test with a single restriction.

Answer 1

Yes, this is possible.

The general multiple linear regression model is

$$y = X\beta + \varepsilon$$

with the $k$ regressor values, along with a constant vector, forming the columns of $X$ and the $p=k+1$ coefficients arranged correspondingly in the vector $\beta$. When there is no collinearity among those columns, the least-squares solution is

$$\hat \beta = (X^\prime X)^{-1}X^\prime y.$$

It fits (or "predicts") the expectations $E[y|X]$ with

$$\hat y = X\hat\beta = X(X^\prime X)^{-1}X^\prime y.$$

Assuming the "errors" $\varepsilon$ are uncorrelated and have common variance $\sigma^2$, the unbiased estimator of $\sigma^2$ is

$$\hat\sigma^2 = \frac{||y - \hat y||^2}{n-p}.$$

(This is the mean squared residual $||y - \hat y||^2/n$ adjusted by the factor $n/(n-p)$ to make the estimator unbiased.) The covariance matrix of the estimates $\hat\beta$ can then be estimated as

$$\operatorname{Cov}(\hat\beta) \approx \hat\sigma^2 (X^\prime X)^{-1}.$$

Any linear combination $c^\prime \beta$ will be estimated with the same linear combination of parameter estimates, $c^\prime \hat\beta$. To compare $c^\prime\beta$ to some predetermined constant $c_0$ (often $0$), compute the variance of the estimate as

$$\operatorname{se}(c^\prime\hat\beta) = \operatorname{Var}(c^\prime\hat\beta) = c^\prime \operatorname{Cov}(\hat\beta)c = \hat\sigma^2 c^\prime(X^\prime X)^{-1}c$$

and conduct a t-test (either one- or two-sided) based on the t statistic

$$t = \frac{c^\prime\hat\beta - c_0}{\sqrt{\operatorname{se}(c^\prime\hat\beta) }}.$$

For $n$ observations, it has $n-p$ degrees of freedom.

---

The following R code shows how to perform these calculations and runs a (fast) simulation demonstrating that one-sided p-values do indeed have a uniform distribution when the null hypothesis holds. The required information is extracted from an lm fit using coef to obtain $\hat \beta$ and vcov for $\hat\sigma^2 (X^\prime X)^{-1}$.

To check the simulation, it conducts a $\chi^2$ test of uniformity; the output for this particular set of 1000 iterations is a p-value of $0.4428$, evidence that the code is performing as claimed.

beta <- c(1,4,2,3)       # The coefficients
n <- 20                  # The number of observations
combo <- c(0,1,1,-1,-1)  # The contrast (starting with an intercept coefficient)
sigma <- 2               # The error SD
n.sim <- 1e3             # Number of iterations of the simulation
#
# Prepare to simulate.
#
set.seed(17)
k <- length(beta)
combo.name <- paste0("Contrast(", paste(combo, collapse=","), ")")
#
# Simulate with many different sets of regressors and responses, but all
# using the same `beta`.
#
p.values <- replicate(n.sim, {
  #
  # Create regressors.
  #
  x <- matrix(rnorm(n*k), n)
  colnames(x) <- paste0("X", 1:k)
  y <- x %*% beta + rnorm(n, 0, sigma)
  #
  # Test the combination.
  #
  fit <- lm(y ~ ., as.data.frame(x))
  beta.hat <- crossprod(coef(fit), combo)
  beta.hat.se <- sqrt(combo %*% vcov(fit) %*% combo)
  t.stat <- beta.hat / beta.hat.se
  p <- pt(t.stat, n-k-1) # A one-sided p-value
  #
  # When running this "for real," uncomment the following lines to
  # see the results.
  #
#   stats <- rbind(coef(summary(fit)), c(beta.hat, beta.hat.se, t.stat, p))
#   rownames(stats)[k+2] <- combo.name
#   print(stats, digits=3)
  #
  # Return the p-value for the (two-sided) t-test
  #
  p
})
#
# Display the simulation results.
#
n.bins <- floor(n.sim / 100)
(p.dist <- chisq.test(table(floor(n.bins*p.values)))$p.value) #$
hist(p.values, freq=FALSE, breaks=n.bins,
     sub=paste("p (uniform) =", format(p.dist, digits=3)))
abline(h = 1, col="Gray", lwd=2, lty=3)

Answer 2

Yes this is indeed very possible, in fact nothing changes. Use the standard F-test provided by whatever software you are using.

Actually, when you only have one parameter the F-test is just the square of a the same t-test. That is, $t^2_{n-k-1} \sim F_{1,n-k-1}$. As Whuber points out in the comments, you have to be carefull in thinking about 1 and 2 sided alternatives. And in this case, (for the above results to be true) you need a two sided alternative. But this should always be the case, unless you have a really good argument for why the statistic cannot be negative (or positive).