2
$\begingroup$

I got this question earlier for a review, but am struggling to find the answer in any texts:

Suppose that you have to fit the model $$y=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i,\,i=1,2,\ldots,20\,,$$ and you want to test the following set of restrictions: $\beta_1=0,\beta_2=2$. Write the matrix $A$ that would allow the restrictions $\beta_1=0,\beta_2=2$ to be written in the form $A\beta = 0$.

The $A$ matrix would be pretty straightforward if there was any equivalence between $\beta_1$ and $\beta_2$, but what we're stuck with is something more like $\beta_1 + \beta_2 - 2 = 0$. All of my book's examples (Montgomery, Peck, Vining's Intro to Linear Regressions) seem to show a matrix $T$ as $T\beta = 0$ or $T\beta = c$ but nothing that could reduce this equation to $T\beta = 0$ from the examples I've seen.

I found some more examples across the internet, but none more explicit than http://home.iitk.ac.in/~shalab/regression/Chapter3-Regression-MultipleLinearRegressionModel.pdf on page 23 (example iv - notation switched to $R\beta = r$). That text also seems to suggest that the equation should be $\beta_1 + \beta_2 = 2$ rather than the above $\beta_1 + \beta_2 - 2 = 0$, which is really confusing me right now.

Any help would be appreciated - thanks.

$\endgroup$
  • $\begingroup$ Because the question imposes two restrictions, the "$0$" on the right hand side of $A\beta=0$ must be a vector of two components. Ergo, since $\beta$ has four components, $A$ must be a $2\times 4$ matrix. To figure out the components of $A,$ all you need to do is apply the definition of matrix multiplication. Then you will discover that the problem cannot be solved in the form given: either you need to augment $\beta$ with another component, modify the meaning of $\beta$ (by subtracting $2$ from $\beta_2$), or replace the vector $0$ with a different vector. $\endgroup$ – whuber Nov 4 '19 at 16:27
  • $\begingroup$ Seems like there's a good chance i'm just struggling with the linear algebra. I understand the 2x4 matrix - most of the examples use something similar (ie β<sub>1</sub> - β<sub>2</sub> = 2 would be T = [0, 1, -1, 0]) but I'm struggling to understand how my 2x4 matrix would have β<sub>2</sub> - 2 = 0. Apologies for my elementary understanding of linear algebra affecting this process. And I can easily see the first row would be [0, 1, 0, 0] = 0, but would the second row then be [0, 0, 2-2/β<sub>2</sub>, 0]? $\endgroup$ – That One Dude Mike Nov 4 '19 at 16:37
  • $\begingroup$ I think this question should have the self-study tag. $\endgroup$ – Michael R. Chernick Nov 4 '19 at 17:09
  • $\begingroup$ Thanks for the heads up, Michael. Long time reader, first time poster, so I definitely added the tag as per your suggestion. Appreciate it! $\endgroup$ – That One Dude Mike Nov 4 '19 at 22:02
  • 1
    $\begingroup$ Here is a MathJax tutorial for typesetting math. $\endgroup$ – StubbornAtom Nov 5 '19 at 20:48
1
$\begingroup$

Because $\mathbb{A}=(a_{ij})$ left-multiplies the four-vector $\beta=(\beta_0, \ldots, \beta_3)^\prime,$ $\mathbb{A}$ must be a $c\times 4$ matrix for some integer $c.$

The definition of matrix multiplication shows that when $0 = (0,0,\ldots,0)^\prime$ has $c$ components, the equation $\mathbb{A}\beta = 0$ is a system of $c$ simultaneous linear equations. The equation for component $i,$ $1\le i\le c,$ is

$$a_{i1}\beta_0 + a_{i2}\beta_1 + a_{i3}\beta_2 + a_{i4}\beta_3 = 0.$$

Among these equations we need to find one that asserts $\beta_1=0$ and another that asserts $\beta_2=2.$ The first assertion involves the linear combination

$$\beta_1 = (0)\beta_0 + (1)\beta_1 + (0)\beta_2 + (0)\beta_3,$$

showing that setting one row of $\mathbb{A}$ to the vector $(0,1,0,0)$ will do the trick.

Unfortunately, the equation $\beta_2=2$ cannot be written as a linear combination of the $\beta_i.$ The problem therefore has no solution.


In practice, there are a few ways to cope with this. One is to put the $2$ on the right hand side. This immediately gives one possible solution:

$$\pmatrix{0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0}\,\beta = \pmatrix{0\\2}.$$

Another is to modify the vector $\beta$ to $(\beta_0, \beta_1, \beta_2 - 2, \beta_3).$ Now--employing the same ideas as before--you can write down a suitable $\mathbb A$ by inspection:

$$\pmatrix{0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0}\,\pmatrix{\beta_0\\\beta_1\\\beta_2-2\\\beta_3} = \pmatrix{0\\0}.$$

This often works with statistical software, because the modification is tantamount to subtracting the values of $x_{i2}$ from the right hand side of the model. To leave the model unchanged, then, you must subtract those values from both sides, giving

$$y_i - 2 x_{i2} = \beta_0 x_{i0} + \beta_1 x_{i1} + (\beta_2-2) x_{i2} + \beta_3 x_{i3} + \varepsilon_i.$$

That is, twice the regressor $x_{i2}$ is subtracted from the response $y_i$ for each observation before fitting the model. In reading its output you will need to remember to add $2$ to its estimate of $\beta_2.$

In models where the $x_{ij}$ are considered to be just numbers--that is, values that are determined or are observed without appreciable error--this does not modify the probabilistic structure of the model, which concerns only the errors $\varepsilon_i.$ Thus, when you fit the model as re-expressed in this way, you can test the hypothesis $\beta_1=0, \beta_2=2$ in the form $\mathbb{A}\beta=0.$

$\endgroup$
  • $\begingroup$ Awesome - thank you! I was wondering if it was possible to add a β<sub>2<sub> term in the A matrix as [1 - ( 2 / β<sub>2</sub> )] or if it needed to be in the β matrix, and you answered my question. Thanks! $\endgroup$ – That One Dude Mike Nov 4 '19 at 19:58
  • $\begingroup$ To be useful in the setting you propose, it is essential that $\mathbb A$ not depend on the data. In particular, (a) it cannot depend on $\beta$ because you don't know $\beta$--you're trying to estimate it--and (b) it cannot depend on the estimates $\hat\beta$ because those are functions of the data. Besides, how would you handle the possibility that $\beta_2=0$? The value $2/\beta_2$ would not exist. $\endgroup$ – whuber Nov 4 '19 at 20:01
  • $\begingroup$ Thank you so much for a complete explanation! $\endgroup$ – That One Dude Mike Nov 4 '19 at 20:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.