2
$\begingroup$

Most statistical methods assume homogeneous (outlier-free) data in which all data points satisfy the same model. However, real data are (not) NEVER homogeneous; and accurate identification of outliers plays an important role in statistical analysis.

Moreover, the OLS classic regression needs to satisfy many assumptions (normality, homoscedasticity of the residuals, linearity of the regression function...).

And with the OLS regression, we have as well to take care of the (multi)colinearity problem between the explanatory variables (measured by the Variance inflation factor (VIF)).

My 2 questions are the following :

1) When we realize the nonparametric MARS (Multivariate adaptive regression splines) regression, do we have to care about the outliers problem, about the (multi)colinearity problem ? And do we have to care about the different assumptions (normality, homoscedasticity of the residuals and linearity of the regression function...) like for OLS regression ? Or when we practice the MARS regression, we don't have to care about all these ?

2) If there are some, what are the conditions of application of MARS ? I have read that there must be lots of observations ? For example, if I have a sample size of n=33 countries and 1 dependent variable and 3 explanatory variables, is it enough for MARS regression ?

Best Regards, looking forward to reading You.

$\endgroup$
3
  • 1
    $\begingroup$ Depending on your goals, OLS may not have to satisfy any assumptions! $\endgroup$
    – Glen
    Dec 9, 2013 at 17:25
  • $\begingroup$ I find the double negative in "(not) NEVER" ambiguous. Do you mean "never" or not? $\endgroup$
    – Glen_b
    Apr 20, 2014 at 11:11
  • $\begingroup$ Hi Glen, what I mean is the real data are NEVER homogeneous or almost never homogeneous. $\endgroup$ May 1, 2014 at 12:38

1 Answer 1

1
$\begingroup$

MARS is a very flexible method and an excellent alternative to OLS regression. Also, it is more interpretable than machine learning methods such as neural networks and SVM. MARS can transform variables, identify higher order interaction between variables which is a huge + vs. OLS.

For Question #1,

  • MARS can handle outliers very well.
  • MARS is distribution free, therefore no assumption. MARS is entirely data driven or in other words it learns from data.
  • When I model using MARS at work, I hardly ever worry about assumptions. However its always a good idea to look at residulas and see if the model fit is adequate and there is no pattern left.

For Question #2,

  • MARS is entirely a data driven method, therefore you might need adequate data to fit MARS. n = 33 is probably a small set for MARS. I would suggest fit both OLS and MARS and see which gives better fit.
$\endgroup$
6
  • $\begingroup$ Many thanks for all your precise responses. But, now I am quite annoyed because I am wondering when or why I should better use MARS regression than high-breakdown regression methods (robust regression like the least median of squares (LMS) or LTS ? Both of them (MARS and robust regression) are useful when there are outliers and when there are some deviations from the classical assumptions from the standard regression model. $\endgroup$ Dec 9, 2013 at 20:38
  • $\begingroup$ @varinsacha I'm not familiar with eithe LMS or LTS. I use MARs when I suspect nonlinearity and higher order interaction. MARS is known to be superior to other regression methods and can easily interpretable. $\endgroup$
    – forecaster
    Dec 9, 2013 at 21:16
  • $\begingroup$ Many Thanks once more for your response. I use the R package "earth" to realize MARS. The big problem with MARS is the same as every nonparametric regression. Indeed nonparametric estimators are much harder to interpret. Are You ok ? Best $\endgroup$ Dec 10, 2013 at 19:53
  • $\begingroup$ @varinsacha, one of the strengts of MARS is it is highly intrepretable vs. other techniques. $\endgroup$
    – forecaster
    Dec 10, 2013 at 20:34
  • $\begingroup$ Hi forecaster, Thanks a lot for all your comments. I will send another message because I have realized a MARS regression with R and I don't understand everything for the interpretation. $\endgroup$ Dec 14, 2013 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.