Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am experiencing some cognitive dissonance about what 'linear in the parameters' means. For example, here and here.

For example, my understanding is $y_i = \beta_0 + \beta_1\beta_2x_1 + \exp(\beta_3)(x_2)^2 + \epsilon$ is not linear in the parameters, because it has two parameter variables multiplied together (namely ${\beta_1, \beta_2}$).

If $\beta_1$ (say) was replaced with $\gamma_1$, a constant, it would be.

Appreciate if someone could clarify this point.

share|improve this question
    
While you Are correct the function is not linear in the parameters, it can be made so with a log transformation – Repmat Mar 18 at 8:42
    
@Repmat How so? How will log transformation help here? – Nick Cox Mar 18 at 13:28
    
I don't see that anything, linear or nonlinear, will make $\beta_1, \beta_2$ separately estimable here. More positively, watch that GLM in different contexts means general linear models and generalized linear models, which overlap but are by no means identical classes. – Nick Cox Mar 18 at 13:29
    
Removing the product and the exp – Repmat Mar 18 at 13:31
1  
OK, so you are reparameterising. That's not transformation (of variables), which was what I was inferring. – Nick Cox Mar 18 at 13:33
up vote 6 down vote accepted

Your example model can be reëxpressed to be linear in the parameters $\alpha=\beta_1\beta_2$ & $\zeta=\exp\beta_3$:

$$g(\operatorname{E} Y) = \beta_0 + \alpha x_1 + \zeta x_2^2$$

(Clearly $\beta_1$ & $\beta_2$ aren't separately estimable; a non-linear model wouldn't help there. And note that $\hat\zeta$ must be constrained to be positive.) Some models can't be so reëxpressed:

$$g(\operatorname{E} Y) = \beta_0 + \beta_1 x_1 + x_2^{\beta_2}$$

Some can be though it's not obvious at first: http://stats.stackexchange.com/a/60504/17230.

There's a very thorough discussion of different meanings of "linear" at How to tell the difference between linear and non-linear regression models?.

share|improve this answer
1  
+1 for the diæresis. – amoeba Mar 18 at 12:52
1  
+1 for the ligature. (In fact I use second-hand tremas, which you can pick up cheap now.) – Scortchi Mar 18 at 12:53
1  
The reëxpression loses the information that $\zeta$ is constrained to be positive. – Juho Kokkala Mar 18 at 15:14
    
@JuhoKokkala: Good point - I'll note that. – Scortchi Mar 18 at 15:16

Linear in the parameters means that you can write your prediction as

$$\beta_0+\sum_{j=1}^px_{ij}\beta_j $$

For some definition of $x_{ij} $. But these x's need not be linear functions of your data. For example, ploynomial fitting of a time series has $x_{ij}=t_i^j $ where $t_i $ is the time associated with data point $i $. The prediction is a non linear function of time, but it is linear in the betas.

UPDATE

In response to the comment, the answer is "sort of". If $\beta_2$ was constant, then the predictor is linear in $\beta_0,\beta_1,\exp (\beta_3) $. It is not linear in $\beta_3$, but a transformation of $\beta_3$. In terms of least squares estimates it doesn't make much difference here.

share|improve this answer
    
Thanks so much for replying. Perhaps my question isn't clarified by including a transformation of the x's. I am asking about the beta's (parameters), not transforms of x's. Perhaps if you could comment on my specific example above. – Ben S Mar 18 at 6:35

I think it's better for you to understand the three components of the GLM. Esp, you need understand how link function is defined.

You can refer to the page 7 in the slides below. 'linear in the parameters' is true after being transformed by the link function.

enter link description here

share|improve this answer
1  
While crucial in general, links don't seem pertinent to this thread. – Nick Cox Mar 18 at 13:38
    
Parameters are linear even before transformation. – Daeyoung Lim Mar 22 at 7:28

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.