# How to identify models as linear or non-linear?

Identify the following models as linear or non-linear. In case of a non-linear model, reduce the model into a linear model by a suitable transformation.

\eqalign{ (a)\quad&y=\beta_0+\beta_1 x+\beta_2 x^2+e \\ (b)\quad&y=\frac{x}{\beta_0+\beta_1 x}+e\\ (c)\quad&y=\frac{\exp(\beta_0+\beta_1 x)}{1+\exp(\beta_0+\beta_1 x)}+e }

where $e\sim\mathcal{N}(0,\sigma^2).$

I know that the model of $(a)$ is linear. I think $(b)$ & $(c)$ are non-linear.How can I make them linear models?

-
Homework question should be tagged as such, because they receive a special treatment. Users who are always ready to assist you probably expect you to show what you tried to solve those questions. – chl Aug 7 '12 at 20:13
Some caution is needed in interpreting this question and its results, because neither (b) nor (c) can be linearized with a transformation. This is because the nonlinear transformations needed to make the parameters enter linearly will affect the error distributions: it will make them non-normal with nonzero expectation and with spreads depending on the values of $x$ (heteroscedasticity) and the parameters (a subtle form of nonlinearity). – whuber Aug 7 '12 at 20:36
@whuber In my answer I was deliberately ignoring the additive error. I was taking the OP to be describing the model by the functional form and made the transformations accordingly. This seemed to be obviously the intent. – Michael Chernick Aug 7 '12 at 20:55

We try to find a re-expression $f$ for which $f(y)$ is a sum of two kinds of things. The first kind is a product of (1) something depending only on the data $x$ and (2) something depending only on the parameters $\beta_i$. The second kind of thing represents "random error." Typically we want the random error, at a minimum, to have an expectation of zero. It's even nicer (and usual to assume) that the error not depend on the data or the parameters and that it have a symmetric distribution.

(I am glossing over one delicate but tangential point: the parameter-dependent part of each term ought itself to be just a constant linear combination of parameters. However, in practice this often does not matter. For instance, although the model $y = \log(\beta)x + e$ depends nonlinearly on the parameter $\beta$, we can easily replace $\beta$ by $\gamma=\log(\beta)$, thereby effecting a reparameterization $y = \gamma x + e$ in which the new parameter $\gamma$ (a) appears in the desired (linear) form and (b) determines everything that could be inferred about the original parameter $\beta$.)

Model $(a)$ is already in this archetypal form, because each non-error term clearly is a product of a parameter and something depending only on the data ($x$ and $x^2$). The error $e$, with a $\mathcal{N}(0,\sigma^2)$ distribution, has an expectation of $0$, is distributed independently of $x$ and the $\beta_i$, and has a symmetric distribution. The point of this example is to emphasize that the dependence on the data can be non-linear, as evidenced by the $x^2$, without affecting the linearity of the model.

Model $(b)$ is not in this form, but we can see such a form in the denominator on the right hand side. Therefore we try to solve for $y$ algebraically, initially ignoring the errors (and making any additional assumptions required to make the algebra work, such as $x\ne 0$):

$$y = \frac{x}{\beta_0 + \beta_1 x} = \frac{1}{\beta_0/x + \beta_1}$$

implies

$$1/y = \beta_1 + \beta_0(1/x).$$

Each of the terms on the right hand side separates as a product of something depending only on the data ($1$ and $1/x$, respectively) and something depending only on the parameters ($\beta_1$ and $\beta_0$, respectively). Typically, when such an expression can be found, it is essentially unique. (You can often shift a constant factor around within each term; e.g., $\beta_0 \times 1/x = 0.5\beta_0 \times 2/x$, but that's an unimportant change.) Evidently, $f(y) = 1/y$ is how $y$ should be re-expressed.

Then, it is important to repeat the calculation with the error term included:

$$1/y = \frac{1}{\frac{x}{\beta_0+\beta_1 x}+e}.$$

This, unfortunately, does not simplify. By assuming the errors are small, we can attempt an approximate simplification by means of an approximation that is linear in the error:

$$1/y \approx \beta_1 + \beta_0/x + \left[-\left(\beta_1 + \beta_0/x\right)^2\right] e + O(e^2).$$

This now is in the desired form, with the error term equal to $-\left[\left(\beta_1 + \beta_0/x\right)^2\right] e$, more or less (neglecting terms of the size of $e^2$, which is typically $\sigma^2$). Because the expectation of $e$ is $0$, the expectation of this error term is (more or less) $0$. However, the dispersion of the error in $1/y$ equals $\left(\beta_1 + \beta_0/x\right)^4 \sigma^2$. This complicated expression depends essentially on both the data and the parameters, leading to a rather complicated--but linear--model. The distribution of this error is no longer normal (in fact, it is asymmetric).

After fitting such a model, one would certainly want to check the assumption required to make the approximation: namely, that $\sigma^2$ is small compared to $y$ for the fitted parameters and for all values of $x$ of interest. If this assumption does not hold, the transformation should be considered unsuccessful and nonlinear modeling (based on the original equation, for which the errors have a particularly nice form) would be advisable.

The analysis of model $(c)$ is similar. The algebraic manipulations change a little, and therefore the answer changes a little, but no new ideas are introduced.

This question is glib in that it blithely ignores these latter complications concerning the error terms. That's probably ok for an elementary, shallow introduction to the ideas of transformation, but it is important to get in the habit of introducing and tracking the error terms in one's models: that is a part of what "statistical thinking" and modeling are all about.

-