1
$\begingroup$

What does it exactly mean by linearity in parameters in the assumptions for classical linear regression model and Gauss-Markov theorem?

My understanding is:

  1. y = b0 + b1 x + e - Linear in parameters
  2. y = b3 + b4^2 x + e - Non-linear in parameters => Can it be re-written like y = b3 + b5 x + e where b5 = b4^2

In case 2, since final values of all b's are nothing but constants ultimately, we can rewrite b4^2 as some other constant b5. So if we rewrite this, case 2 also becomes linear in parameters. Isn't it?

e.g

  1. y = 10 + 16 x + e (Linear in parameter)
  2. y = 10 + 5^2 x + e (Non-linear in parameter) but can be re-written as y = 10 + 25 x + e which is linear in parameter.

Please help me to clarify what I'm lacking to undestand.

Thanks.

$\endgroup$
  • 1
    $\begingroup$ Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Similar questions were asked on other stackexchange sites, see here and here. $\endgroup$ – T.E.G. May 14 '17 at 5:20
3
$\begingroup$

First thing is that we're essentially dealing with a vector of parameters, so that we can consider that we have $\boldsymbol{\beta} = (\beta_0,\beta_1,...,\beta_p)^\top$.

See https://en.wikipedia.org/wiki/Linear_algebra and/or https://en.wikipedia.org/wiki/Linear_map and notice (working in the notation of the second link, and writing $\sum_{j=0}^p \beta_j x_j$ as $X\boldsymbol{\beta}$), that $f(\boldsymbol{\beta})=E(Y|x)=X\boldsymbol{\beta}$ satisfies the two conditions in the definition of a linear map: $f(\boldsymbol{\beta}_1+\boldsymbol{\beta}_2)=f(\boldsymbol{\beta}_1)+f(\boldsymbol{\beta}_2)$ and for some scalar constant $\alpha$, that $f(\alpha\boldsymbol{\beta})=\alpha f(\boldsymbol{\beta})$.

If it is not immediately clear (or if you're unused to working with vectors and matrices), you can check those conditions for each of your example models.

For example your first model has $E(Y|x) = \beta_0\cdot 1+\beta_1 \cdot x = f(\beta_0,\beta_1)$. Now consider

\begin{eqnarray} f((\beta_0+\gamma_0),(\beta_1+\gamma_1))&=& (\beta_0+\gamma_0)\cdot 1+(\beta_1+\gamma_1) \cdot x \\ &=& \beta_0\cdot 1+\beta_1 \cdot x + \gamma_0\cdot 1+\gamma_1 \cdot x\\ &=&f(\beta_0,\beta_1)+f(\gamma_0,\gamma_1) \end{eqnarray}

That's the first condition; you can check the second condition in a similar way.

No doubt that seems trivial to the point of banality - but once you have linearity, a lot follows from that. Linear algebra underlies a large number of widely used statistical models and it's worth learning some of the basics before trying to understand a theorem like Gauss-Markov.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.