Multiple Linear Regression Coefficient Estimators and Hypothesis Testing

Question

A multiple linear regression model is considered. It is assumed that $$ Y_i = \beta_1x_{i1} + \beta_2x_{i2} + \beta_3x_{i3} + \epsilon_i$$ where $\epsilon$-s are independent and have the same normal distribution with zero expectation and unknown variance $\sigma^2$. 100 measurements are made, i.e $i = 1,2,..., 100.$ The explanatory variables take the following values: $x_{i1} = 2$ for $1 \leq i \leq 25$ and $0$ otherwise, $x_{i2} = \sqrt{2}$ for $26 \leq i \leq 75$ and $0$ otherwise, $x_{i3} = 2$ for $76 \leq i \leq 100$ and $0$ otherwise.

a) Let $\hat{\beta_1},\hat{\beta_2}, \hat{\beta_3}$ be least squares estimators of $\beta_1, \beta_2, \beta_3$. Prove that in the considered case $\hat{\beta_1},\hat{\beta_2}, \hat{\beta_3}$ are independent, and $$Var(\hat{\beta_1}) = Var(\hat{\beta_3}) = Var(\hat{\beta_3})$$ Do these properties hold in the general case? If not, give counterexamples.

b) Perform a test for $$H_0: \beta_1 + \beta_3 = 2\beta_2$$vs.$$H_1: \beta_1 + \beta_3 \neq 2\beta_2$$ The significance level is 0.05. The least squares estimates of $\beta_1, \beta_2$ and $\beta_3$ are $0.9812, 1.8851$ and $3.4406$, respectively. The unbiased estimate of the variance $\sigma^2$ is $3.27$.

For a) I know the OLS estimator for $\hat{\beta} = (X^TX)^{-1}X^Ty$, and $Var(\hat{\beta}) = \sigma^2 (X^TX)^{-1}$. But I don't know how to attain explicit expressions for each of the coefficients from this. Although it seems quite clear that the estimators are independent, for instance $P(\hat{\beta_3} = \beta_3, \hat{\beta_1} = 0, \hat{\beta_2} = 0) = P(\hat{\beta_3} = \beta_3)$ but I don't how to write a proper proof. I believe the estimators are generally dependent and have unequal variance, but can't come up with any particular examples.

For b) not sure what test-statistic to use (t or F) and how to set it up. Also don't know the standard errors of the coefficients

Answer 1

You have already given the formula for the variance matrix for the coefficient estimator. The Gramian matrix of the design matrix for the regression ---which appears in that formula--- is:

$$\begin{aligned} \mathbf{x}^\text{T} \mathbf{x} &= \begin{bmatrix} \mathbf{x}_1 \cdot \mathbf{x}_1 & \mathbf{x}_1 \cdot \mathbf{x}_2 & \mathbf{x}_1 \cdot \mathbf{x}_3 \\ \mathbf{x}_2 \cdot \mathbf{x}_1 & \mathbf{x}_2 \cdot \mathbf{x}_2 & \mathbf{x}_2 \cdot \mathbf{x}_3 \\ \mathbf{x}_3 \cdot \mathbf{x}_1 & \mathbf{x}_3 \cdot \mathbf{x}_2 & \mathbf{x}_3 \cdot \mathbf{x}_3 \\ \end{bmatrix} \\[6pt] &= \begin{bmatrix} \sum_{i=1}^{25} 2 \times 2 & 0 & 0 \\ 0 & \sum_{i=26}^{75} \sqrt{2} \times \sqrt{2} & 0 \\ 0 & 0 & \sum_{i=76}^{100} 2 \times 2 \\ \end{bmatrix} \\[6pt] &= \begin{bmatrix} 25 \times 4 & 0 & 0 \\ 0 & 50 \times 2 & 0 \\ 0 & 0 & 25 \times 4 \\ \end{bmatrix} \\[6pt] &= 100 \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix}. \\[6pt] \end{aligned}$$

This is proportionate to the identity matrix, which means that the coefficient estimators are uncorrelated with equal variance. You therefore have:

$$\mathbb{V}(\hat{\boldsymbol{\beta}}) = \sigma^2 (\mathbf{x}^\text{T} \mathbf{x})^{-1} = \frac{\sigma^2}{100} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix}. \\[6pt] $$

You should be able to derive the coefficient estimates from the standard regression formula and then use these to formulate the hypothesis tests. Each hypothesis test is testing a linear combination of the coefficients, so you can use the rules for linear combinations of normal random variables to derive the standard errors of the test statistics.

Answer 2

As You know that the OLS estimator is a linear function of $y$. $\hat{\beta} \sim N(\beta, \sigma^2(X’X)^{-1})$ since $\epsilon \sim N(0,\sigma^2)$.

1) All you need to show is that the matrix $(X’X)^{-1}$ is a scalar matrix. Just compute the inverse using appropriate entries for $x_1,x_2,x_3$

2) All you need to do is test the following Hypothesis :-

$H_0 : l’{\beta} = 0$ vs $H_1 : l’{\beta} \neq 0$

Where $l’$ is (1,-2,1).

Now use a $t$ test on $l’\hat{\beta}$ which follows a univariate normal distribution.