7
$\begingroup$

I am experiencing some cognitive dissonance about what 'linear in the parameters' means. For example, here and here.

For example, my understanding is $y_i = \beta_0 + \beta_1\beta_2x_1 + \exp(\beta_3)(x_2)^2 + \epsilon$ is not linear in the parameters, because it has two parameter variables multiplied together (namely ${\beta_1, \beta_2}$).

If $\beta_1$ (say) was replaced with $\gamma_1$, a constant, it would be.

Appreciate if someone could clarify this point.

$\endgroup$
  • $\begingroup$ While you Are correct the function is not linear in the parameters, it can be made so with a log transformation $\endgroup$ – Repmat Mar 18 '16 at 8:42
  • $\begingroup$ @Repmat How so? How will log transformation help here? $\endgroup$ – Nick Cox Mar 18 '16 at 13:28
  • $\begingroup$ I don't see that anything, linear or nonlinear, will make $\beta_1, \beta_2$ separately estimable here. More positively, watch that GLM in different contexts means general linear models and generalized linear models, which overlap but are by no means identical classes. $\endgroup$ – Nick Cox Mar 18 '16 at 13:29
  • $\begingroup$ Removing the product and the exp $\endgroup$ – Repmat Mar 18 '16 at 13:31
  • 1
    $\begingroup$ OK, so you are reparameterising. That's not transformation (of variables), which was what I was inferring. $\endgroup$ – Nick Cox Mar 18 '16 at 13:33
6
$\begingroup$

Your example model can be reëxpressed to be linear in the parameters $\alpha=\beta_1\beta_2$ & $\zeta=\exp\beta_3$:

$$g(\operatorname{E} Y) = \beta_0 + \alpha x_1 + \zeta x_2^2$$

(Clearly $\beta_1$ & $\beta_2$ aren't separately estimable; a non-linear model wouldn't help there. And note that $\hat\zeta$ must be constrained to be positive.) Some models can't be so reëxpressed:

$$g(\operatorname{E} Y) = \beta_0 + \beta_1 x_1 + x_2^{\beta_2}$$

Some can be though it's not obvious at first: https://stats.stackexchange.com/a/60504/17230.

There's a very thorough discussion of different meanings of "linear" at How to tell the difference between linear and non-linear regression models?.

$\endgroup$
  • 1
    $\begingroup$ The reëxpression loses the information that $\zeta$ is constrained to be positive. $\endgroup$ – Juho Kokkala Mar 18 '16 at 15:14
  • $\begingroup$ @JuhoKokkala: Good point - I'll note that. $\endgroup$ – Scortchi - Reinstate Monica Mar 18 '16 at 15:16
5
$\begingroup$

Linear in the parameters means that you can write your prediction as

$$\beta_0+\sum_{j=1}^px_{ij}\beta_j $$

For some definition of $x_{ij} $. But these x's need not be linear functions of your data. For example, ploynomial fitting of a time series has $x_{ij}=t_i^j $ where $t_i $ is the time associated with data point $i $. The prediction is a non linear function of time, but it is linear in the betas.

UPDATE

In response to the comment, the answer is "sort of". If $\beta_2$ was constant, then the predictor is linear in $\beta_0,\beta_1,\exp (\beta_3) $. It is not linear in $\beta_3$, but a transformation of $\beta_3$. In terms of least squares estimates it doesn't make much difference here.

$\endgroup$
  • $\begingroup$ Thanks so much for replying. Perhaps my question isn't clarified by including a transformation of the x's. I am asking about the beta's (parameters), not transforms of x's. Perhaps if you could comment on my specific example above. $\endgroup$ – Ben S Mar 18 '16 at 6:35
0
$\begingroup$

I think it's better for you to understand the three components of the GLM. Esp, you need understand how link function is defined.

You can refer to the page 7 in the slides below. 'linear in the parameters' is true after being transformed by the link function.

enter link description here

$\endgroup$
  • 1
    $\begingroup$ While crucial in general, links don't seem pertinent to this thread. $\endgroup$ – Nick Cox Mar 18 '16 at 13:38
  • $\begingroup$ Parameters are linear even before transformation. $\endgroup$ – Daeyoung Lim Mar 22 '16 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.