4
$\begingroup$

There exist many techniques that can be used for regression task. Some are from statistics (e.g. generalized least squares) and some from machine learning (e.g. MLP), some are linear and some nonlinear, some are parametric and some nonparametric. Are there available any categorization schemes of regression techniques?

$\endgroup$
2
  • 1
    $\begingroup$ Are you seeking a taxonomy of predictive modeling methods for univariate $Y$? $\endgroup$ Jul 27, 2013 at 12:00
  • 1
    $\begingroup$ Yes, this is what I am exactly searching for. $\endgroup$
    – sitems
    Jul 27, 2013 at 12:04

1 Answer 1

2
$\begingroup$

In the regression modeling realm here is a list.

  1. Linear models with assumed normal residuals (OLS)
  2. generalized linear models (e.g., gamma regression)
  3. generalized additive models
  4. transform-both-sides nonparametric additive regression (AVAS, ACE, canonical variates with splines)
  5. quantile regression
  6. other robust regression methods (see Venables & Ripley book)
  7. semiparametric methods based on the ranks of $Y$ (Cox proportional odds model, proportional odds model, the larger family of cumulative probability models)
  8. Penalized additive regression methods: lasso, quadratic penalty, elastic net

For # 7 predictions are things like log relative hazard or log relative odds. The predictions can be transformed into predicted quantiles or means.

$\endgroup$
3
  • $\begingroup$ It looks good, but there are many techniques missing (especially machine learning), for example SVM, random forests, adaboost, MLP, MARS, ... $\endgroup$
    – sitems
    Jul 27, 2013 at 12:40
  • $\begingroup$ Yes that's why I prefaced it with 'regression modeling realm'. But I've edited the list to include the additive regression methods used in machine learning. $\endgroup$ Jul 27, 2013 at 12:46
  • $\begingroup$ Anyway, it should be clear from such categorization, which techniques are linear and nonlinear and which parametric and nonparametric. $\endgroup$
    – sitems
    Jul 27, 2013 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.