1
$\begingroup$

I've come across this problem of estimating parameters of nonlinear model using closed-form solution. I've read that logarithms can be used for certain forms, but they can't be applied to everything. For example, how can a function of form

$y=\beta_1x/(\beta_2x+\beta_3)$

be linearized so the closed-form solution can be found? Taking inverse gets me to

$1/y=\beta_2/\beta_1+\beta_3/(\beta_1x)=a+b/x$

but that does not seem to lead anywhere. If I now apply the closed-form solution I get only two parameters.I can't solve all the three parameters of the original equation with them, can I? Could someone clarify this transformation/linearization process?

$\endgroup$
2
$\begingroup$
  1. No transformation can "solve" this problem of nonidentifiability, it's inherent.

    Look at your original equation and choose values for the three coefficients (say $\beta_1=1, \beta_2=2, \beta_3=3$ for example). Now double them all, and note that the factor of 2 cancels in numerator and denominator -- two different sets of coefficients specify the exact same model, so data cannot choose between them; indeed there's an infinity of solutions for any given set of data. You have a model with two degrees of freedom and three parameters; it's not appreciably different from a model like "$y = (\alpha_1-\alpha_2)x$" (two parameters but only a single degree of freedom; in that case, any model with $(\alpha_1,\alpha_2)$ is the same as $(\alpha_1+k,\alpha_2+k)$ for any $k$).

  2. A more general point: Since such equations are intended to represent relationships between data, the actual relationship isn't error free.

    When you write $y=\beta_1x/(\beta_2x+\beta_3)$ you don't literally mean that (and you should not say what you don't mean). If you look at your $y$ and $x$ values they don't all lie exactly on the curve, so you're writing "=" when you do not have equality; you have no business calling those things equal. You might mean that there's an underlying relationship of that form, but you don't observe the underlying relationship, only more-or-less noisy data.

    The part of the model that accounts for how $y$ deviates from $\beta_1x/(\beta_2x+\beta_3)$ is missing in such equations - the noise term. The way in which that enters the model is critical (and models with two different error terms may be quite different when you transform the relationship).

    Let's start by removing the nonidentifiability (this can be done in several ways but I'll just choose one):

    $$y = \frac{x}{\frac{\beta_2}{\beta_1} x+\frac{\beta_3}{\beta_1}}=\frac{x}{\gamma_2 x+{\gamma_3}}$$

    But as I said, this is missing how the observed $y$ deviates from the RHS.

    Consider these three models:

    $$y_i = \frac{x_i}{\gamma_2 x_i+{\gamma_3}}+\eta_i$$

    $$y_i = \frac{x_i}{\gamma_2x_i +{\gamma_3}+x_i\epsilon_i}$$

    $$y_i = \frac{x_i}{\gamma_2x_i +{\gamma_3}+\xi_i}$$

    where $\eta_i$ and $\epsilon_i$ and $\xi$ are each sets of independent identically distributed error terms. There are many other possibilities for how noise might come in there of course.

Here are plots of random samples from each of these models, and the deviations of the points from the (same) underlying curve in each case.

Plots of simulated data (y vs x) from the three equations, and plots of the noise term against x

[Even though the dashed lines may appear to be symmetric about the curve (or about 0 for the plots in the second row) they aren't for the second and third model.]

As we see the effect of the different noise terms can be different in different parts of the relationship. The first has constant spread of the noise for all $x$ and the spread in the second one starts of small but rapidly increases then levels off to approach a constant variance for larger $x$. The spread of the noise in the third starts off very large but continually shrinks as $x$ increases, becoming quite small near the end.

Now consider what happens when you transform the relationship as you did in the question. One equation ends up with a constant-variance additive error term but the other two don't. This means that the strategy of recasting in terms of $\frac{1}{y_i}$ will only work in one of those cases.

However, we can find ways to suitably estimate parameters with all three equations.

The first equation is one you'd apply nonlinear least squares to.

The second you'd transform (as you did in your question) to:

$$\frac{1}{y_i} = {\gamma_2 +{\gamma_3}\frac{1}{x_i}+\epsilon_i}$$

at which point you could use ordinary least squares, regressing $\frac{1}{y_i}$ on $\frac{1}{x_i}$.

The third you'd transform to:

$$ \frac{x_i}{y_i} = \gamma_2x_i +{\gamma_3}+\xi_i $$

where you'd use ordinary least squares, regressing $\frac{x_i}{y_i}$ on $x_i$.

Each had the exact same form without noise but different forms once noise was considered. Even though we're estimating the same coefficients, the way those with-noise models imply you should estimate them are quite different (for example, estimation in the second model puts relatively less weight on a point with a large x-value than the third one does)

The error term is part of you model and it matters how you write it. The form of the error often merits as much careful consideration as the form of the relationship for the mean. Often it's also important to consider whether error in the $x$-variable is present (and sufficiently large that it cannot simply be ignored). If it's not very small compared to that in the response, it may be just as important as the other components in deciding a good way to deal with such an equation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.