2
$\begingroup$

I am trying to find examples of datasets that cannot be modeled well using linear models.

I get that, e.g., polynomials cannot model data (say the likes of y=1/x) that settles or converges to a finite value, since polynomials " escape" to infinity at $\pm \infty$. So I have seen recommended to use models of the type :$$Ax^{b}$$ for said datasets. Question 1: Given that linearity is only required on the parameters ( and not on the variable(s)), why can't we use linear approximations to model these sets? Question 2 Any other example(s) of data sets that cannot be modeled linearly ( meaning with generalized linear regression), preferably with some argument?

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you mean some kind of series expansion? $\endgroup$
    – Dave
    Commented Oct 30, 2020 at 1:39
  • $\begingroup$ Not sure I understood, Dave. Looking for examples of data sets for which $y=a_0+a_1\beta_1+a_2 \beta2+....+a_n\beta_n$, where $a_i$ are constant and $\beta_i$ are functions does not provide a good fit. If y were a polynomial, an example would be a data set whose y values approach 0, since polynomials will approach $\pm \infty$ as x values go to $\pm \infty$. Sorry if I was not clear. $\endgroup$
    – MSIS
    Commented Oct 30, 2020 at 3:03

1 Answer 1

Reset to default
3
$\begingroup$

Let any relationship

$$y = f(\mathbf x),\qquad \mathbf x= (x_1,...x_k)$$

Then the first-order Taylor expansion around some point $\mathbf x^0$gives

$$y = f(\mathbf x^0) + \frac{\partial f(\mathbf x^0)}{\partial x_1}(x_1 - x_1^0)+\dots+\frac{\partial f(\mathbf x^0)}{\partial x_1}(x_1 - x_1^0)+\dots R$$

Re-arrange

$$y = \left [f(\mathbf x^0) - \sum_{j=1}^{k}\frac{\partial f(\mathbf x^0)}{\partial x_j}x_j^0\right]\,+\,\frac{\partial f(\mathbf x^0)}{\partial x_1}x_1 +\dots +\frac{\partial f(\mathbf x^0)}{\partial x_k}x_k\,+\,R$$

The correspondence with the $a + b_1x_1 +...$ polynomial is obvious.

Going for the "translog" specification, will give you a second-order approximation to any function.

As for $y=Ax^b$ it is transformed into a polynomial by taking logarithms, $\ln y = \ln A + bx$, at least when positive variables are involved.

The question with these approximations is whether they are adequate for the purpose at hand. The fact that they may not capture correctly the whole range of the relation, may not be of consequence, if what you are interested is the characterization of the relationship in the "center". But if you are interested in characterizing the extremes, then more elaborate constructs may be required. So the issue of choosing a functional form is not so much linked to "data set structure", but to the purposes of the research.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.