Identify the following models as linear or non-linear. In case of a non-linear model, reduce the model into a linear model by a suitable transformation.

$$\eqalign{ (a)\quad&y=\beta_0+\beta_1 x+\beta_2 x^2+e \\ (b)\quad&y=\frac{x}{\beta_0+\beta_1 x}+e\\ (c)\quad&y=\frac{\exp(\beta_0+\beta_1 x)}{1+\exp(\beta_0+\beta_1 x)}+e }$$

where $e\sim\mathcal{N}(0,\sigma^2).$

I know that the model of $(a)$ is linear. I think $(b)$ & $(c)$ are non-linear.How can I make them linear models?

  • 2
    $\begingroup$ Homework question should be tagged as such, because they receive a special treatment. Users who are always ready to assist you probably expect you to show what you tried to solve those questions. $\endgroup$ – chl Aug 7 '12 at 20:13
  • $\begingroup$ Some caution is needed in interpreting this question and its results, because neither (b) nor (c) can be linearized with a transformation. This is because the nonlinear transformations needed to make the parameters enter linearly will affect the error distributions: it will make them non-normal with nonzero expectation and with spreads depending on the values of $x$ (heteroscedasticity) and the parameters (a subtle form of nonlinearity). $\endgroup$ – whuber Aug 7 '12 at 20:36
  • $\begingroup$ @whuber In my answer I was deliberately ignoring the additive error. I was taking the OP to be describing the model by the functional form and made the transformations accordingly. This seemed to be obviously the intent. $\endgroup$ – Michael R. Chernick Aug 7 '12 at 20:55

We try to find a re-expression $f$ for which $f(y)$ is a sum of two kinds of things. The first kind is a product of (1) something depending only on the data $x$ and (2) something depending only on the parameters $\beta_i$. The second kind of thing represents "random error." Typically we want the random error, at a minimum, to have an expectation of zero. It's even nicer (and usual to assume) that the error not depend on the data or the parameters and that it have a symmetric distribution.

(I am glossing over one delicate but tangential point: the parameter-dependent part of each term ought itself to be just a constant linear combination of parameters. However, in practice this often does not matter. For instance, although the model $y = \log(\beta)x + e$ depends nonlinearly on the parameter $\beta$, we can easily replace $\beta$ by $\gamma=\log(\beta)$, thereby effecting a reparameterization $y = \gamma x + e$ in which the new parameter $\gamma$ (a) appears in the desired (linear) form and (b) determines everything that could be inferred about the original parameter $\beta$.)

Model $(a)$ is already in this archetypal form, because each non-error term clearly is a product of a parameter and something depending only on the data ($x$ and $x^2$). The error $e$, with a $\mathcal{N}(0,\sigma^2)$ distribution, has an expectation of $0$, is distributed independently of $x$ and the $\beta_i$, and has a symmetric distribution. The point of this example is to emphasize that the dependence on the data can be non-linear, as evidenced by the $x^2$, without affecting the linearity of the model.

Model $(b)$ is not in this form, but we can see such a form in the denominator on the right hand side. Therefore we try to solve for $y$ algebraically, initially ignoring the errors (and making any additional assumptions required to make the algebra work, such as $x\ne 0$):

$$y = \frac{x}{\beta_0 + \beta_1 x} = \frac{1}{\beta_0/x + \beta_1}$$


$$1/y = \beta_1 + \beta_0(1/x).$$

Each of the terms on the right hand side separates as a product of something depending only on the data ($1$ and $1/x$, respectively) and something depending only on the parameters ($\beta_1$ and $\beta_0$, respectively). Typically, when such an expression can be found, it is essentially unique. (You can often shift a constant factor around within each term; e.g., $\beta_0 \times 1/x = 0.5\beta_0 \times 2/x$, but that's an unimportant change.) Evidently, $f(y) = 1/y$ is how $y$ should be re-expressed.

Then, it is important to repeat the calculation with the error term included:

$$1/y = \frac{1}{\frac{x}{\beta_0+\beta_1 x}+e}.$$

This, unfortunately, does not simplify. By assuming the errors are small, we can attempt an approximate simplification by means of an approximation that is linear in the error:

$$1/y \approx \beta_1 + \beta_0/x + \left[-\left(\beta_1 + \beta_0/x\right)^2\right] e + O(e^2).$$

This now is in the desired form, with the error term equal to $-\left[\left(\beta_1 + \beta_0/x\right)^2\right] e$, more or less (neglecting terms of the size of $e^2$, which is typically $\sigma^2$). Because the expectation of $e$ is $0$, the expectation of this error term is (more or less) $0$. However, the dispersion of the error in $1/y$ equals $\left(\beta_1 + \beta_0/x\right)^4 \sigma^2$. This complicated expression depends essentially on both the data and the parameters, leading to a rather complicated--but linear--model. The distribution of this error is no longer normal (in fact, it is asymmetric).

After fitting such a model, one would certainly want to check the assumption required to make the approximation: namely, that $\sigma^2$ is small compared to $y$ for the fitted parameters and for all values of $x$ of interest. If this assumption does not hold, the transformation should be considered unsuccessful and nonlinear modeling (based on the original equation, for which the errors have a particularly nice form) would be advisable.

The analysis of model $(c)$ is similar. The algebraic manipulations change a little, and therefore the answer changes a little, but no new ideas are introduced.

This question is glib in that it blithely ignores these latter complications concerning the error terms. That's probably ok for an elementary, shallow introduction to the ideas of transformation, but it is important to get in the habit of introducing and tracking the error terms in one's models: that is a part of what "statistical thinking" and modeling are all about.

  • $\begingroup$ I deleted my answer. But now this is an answer to both parts a) and b) of the question. I of course get the point you make about the effect of the transformation on the additive error term. However, I am not sure that the problem necessarily meant to have an additive error term. i think it is there mainly to suggest that the model is observed with error without a particular concern for the form of the error term. Also we went into a logn discussion about my answer and whetehr or not I should take it down and then you put back up 2/3rds of that answer. $\endgroup$ – Michael R. Chernick Aug 7 '12 at 22:28
  • $\begingroup$ @Michael I offered this reply as an example of how one might address questions that look like homework but are not explicitly tagged as such. It is here only as a reaction to your now-deleted post. As a moderator, rather than downvoting low-quality replies, I prefer to provide constructive examples of what a good answer might look like (understanding that my efforts are never perfect and can always be improved: take that as an invitation to post an even better answer!). $\endgroup$ – whuber Aug 7 '12 at 22:33
  • $\begingroup$ It just appears that I was being criticized for answering a homework type problem and essentially asked to delete it and then you give a partial answer to the same question. I see nothing particularly low quality about my reply. It was a complete correct answwer to the question. As I said although the answer ignores the additive noise component the additive nature of the noise was not a serious part of the question. I have seen cases where the noise term was deliberately specified as multiplicative just so it would be additive after the log transformation. $\endgroup$ – Michael R. Chernick Aug 8 '12 at 0:16
  • $\begingroup$ The intent of the problem was to transform to a linear model. $\endgroup$ – Michael R. Chernick Aug 8 '12 at 0:17
  • $\begingroup$ @MichaelChernick: You are absolute right . Thank you for help. $\endgroup$ – Argha Aug 8 '12 at 5:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.