What $A$ would allow $\beta_1=0,\beta_2 = 2$ to be written in the form $A\beta = 0$?

Question

I got this question earlier for a review, but am struggling to find the answer in any texts:

Suppose that you have to fit the model $$y=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i,\,i=1,2,\ldots,20\,,$$ and you want to test the following set of restrictions: $\beta_1=0,\beta_2=2$. Write the matrix $A$ that would allow the restrictions $\beta_1=0,\beta_2=2$ to be written in the form $A\beta = 0$.

The $A$ matrix would be pretty straightforward if there was any equivalence between $\beta_1$ and $\beta_2$, but what we're stuck with is something more like $\beta_1 + \beta_2 - 2 = 0$. All of my book's examples (Montgomery, Peck, Vining's Intro to Linear Regressions) seem to show a matrix $T$ as $T\beta = 0$ or $T\beta = c$ but nothing that could reduce this equation to $T\beta = 0$ from the examples I've seen.

I found some more examples across the internet, but none more explicit than http://home.iitk.ac.in/~shalab/regression/Chapter3-Regression-MultipleLinearRegressionModel.pdf on page 23 (example iv - notation switched to $R\beta = r$). That text also seems to suggest that the equation should be $\beta_1 + \beta_2 = 2$ rather than the above $\beta_1 + \beta_2 - 2 = 0$, which is really confusing me right now.

Any help would be appreciated - thanks.

Answer 1

Because $\mathbb{A}=(a_{ij})$ left-multiplies the four-vector $\beta=(\beta_0, \ldots, \beta_3)^\prime,$ $\mathbb{A}$ must be a $c\times 4$ matrix for some integer $c.$

The definition of matrix multiplication shows that when $0 = (0,0,\ldots,0)^\prime$ has $c$ components, the equation $\mathbb{A}\beta = 0$ is a system of $c$ simultaneous linear equations. The equation for component $i,$ $1\le i\le c,$ is

$$a_{i1}\beta_0 + a_{i2}\beta_1 + a_{i3}\beta_2 + a_{i4}\beta_3 = 0.$$

Among these equations we need to find one that asserts $\beta_1=0$ and another that asserts $\beta_2=2.$ The first assertion involves the linear combination

$$\beta_1 = (0)\beta_0 + (1)\beta_1 + (0)\beta_2 + (0)\beta_3,$$

showing that setting one row of $\mathbb{A}$ to the vector $(0,1,0,0)$ will do the trick.

Unfortunately, the equation $\beta_2=2$ cannot be written as a linear combination of the $\beta_i.$ The problem therefore has no solution.

---

In practice, there are a few ways to cope with this. One is to put the $2$ on the right hand side. This immediately gives one possible solution:

$$\pmatrix{0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0}\,\beta = \pmatrix{0\\2}.$$

Another is to modify the vector $\beta$ to $(\beta_0, \beta_1, \beta_2 - 2, \beta_3).$ Now--employing the same ideas as before--you can write down a suitable $\mathbb A$ by inspection:

$$\pmatrix{0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0}\,\pmatrix{\beta_0\\\beta_1\\\beta_2-2\\\beta_3} = \pmatrix{0\\0}.$$

This often works with statistical software, because the modification is tantamount to subtracting the values of $x_{i2}$ from the right hand side of the model. To leave the model unchanged, then, you must subtract those values from both sides, giving

$$y_i - 2 x_{i2} = \beta_0 x_{i0} + \beta_1 x_{i1} + (\beta_2-2) x_{i2} + \beta_3 x_{i3} + \varepsilon_i.$$

That is, twice the regressor $x_{i2}$ is subtracted from the response $y_i$ for each observation before fitting the model. In reading its output you will need to remember to add $2$ to its estimate of $\beta_2.$

In models where the $x_{ij}$ are considered to be just numbers--that is, values that are determined or are observed without appreciable error--this does not modify the probabilistic structure of the model, which concerns only the errors $\varepsilon_i.$ Thus, when you fit the model as re-expressed in this way, you can test the hypothesis $\beta_1=0, \beta_2=2$ in the form $\mathbb{A}\beta=0.$