Some basic models, like multiple linear regression (MLR), are parametric but usually limited to linear interactions between X and y. Other models can represent non-linear interactions between X and y, but to my understanding, most of these are non-parametric, such as Random Forest. I wonder if there are parametric models capable of embedding non-linear behaviour. I thought of some model that would assume polynomials in its structure, but I did not find anything.

  • 5
    $\begingroup$ Additive models? Linear regression with splines? Neural networks? “Nonparametric” statistics lacks a clear definition. (I thought Cross Validated’s New Year’s Resolution was to get rid of the term.) // You might be interested in the video by MathematicalMonk (Jeff Miller) about nonlinear basis functions in linear regression. $\endgroup$
    – Dave
    Dec 1 '21 at 13:12
  • $\begingroup$ Sorry about the non-parametric term, I was not aware it was not suitable to use it. To be fair, my question derives from a need of extrapolating a model to a unknown range of inputs. I originally used random forests but the threshold-method they use makes it insensitive to extrapolation. An alternative I thought of was of using some sort of multiple linear regression that could embed nonlinear behaviour. I am aware of the many limitations of such an approach, but I wanted to at least check the behaviour with a partial dependence plot for preliminary exploration. $\endgroup$
    – Henrique
    Dec 1 '21 at 14:24
  • $\begingroup$ Quadratic Discriminant Analysis $\endgroup$
    – Emre Toner
    Dec 1 '21 at 22:35
  • $\begingroup$ Polynomial regression? $\endgroup$
    – Firebug
    Dec 1 '21 at 22:44
  • 1
    $\begingroup$ The more flexible the model, the more problematic extrapolation becomes. That doesn't mean you can't extrapolate from an additive model, Gaussian process model, etc., etc., but you have to be extremely careful. It might be better to re-focus your question (or ask another question) specifically about using extrapolation with flexible (nonlinear and/or 'modern nonparametric') $\endgroup$
    – Ben Bolker
    Dec 1 '21 at 22:50

You can transform your original predictors in different ways, e.g., using splines or sine/cosine transformations, then feed them into regression-type models (OLS, GAMs, ...). Frank Harrell's Regression Modeling Strategies discusses this at length.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.