1
$\begingroup$

Most statistical methods assume homogeneous (outlier-free) data in which all data points satisfy the same model. However, real data are (not) NEVER homogeneous; and accurate identification of outliers plays an important role in statistical analysis.

Moreover, the OLS classic regression needs to satisfy many assumptions (normality, homoscedasticity of the residuals, linearity of the regression function...).

And with the OLS regression, we have as well to take care of the (multi)colinearity problem between the explanatory variables (measured by the Variance inflation factor (VIF)).

My 2 questions are the following :

1) When we realize the nonparametric MARS (Multivariate adaptive regression splines) regression, do we have to care about the outliers problem, about the (multi)colinearity problem ? And do we have to care about the different assumptions (normality, homoscedasticity of the residuals and linearity of the regression function...) like for OLS regression ? Or when we practice the MARS regression, we don't have to care about all these ?

2) If there are some, what are the conditions of application of MARS ? I have read that there must be lots of observations ? For example, if I have a sample size of n=33 countries and 1 dependent variable and 3 explanatory variables, is it enough for MARS regression ?

Best Regards, looking forward to reading You.

$\endgroup$
  • 1
    $\begingroup$ Depending on your goals, OLS may not have to satisfy any assumptions! $\endgroup$ – Glen Dec 9 '13 at 17:25
  • $\begingroup$ I find the double negative in "(not) NEVER" ambiguous. Do you mean "never" or not? $\endgroup$ – Glen_b Apr 20 '14 at 11:11
  • $\begingroup$ Hi Glen, what I mean is the real data are NEVER homogeneous or almost never homogeneous. $\endgroup$ – varin sacha May 1 '14 at 12:38
1
$\begingroup$

MARS is a very flexible method and an excellent alternative to OLS regression. Also, it is more interpretable than machine learning methods such as neural networks and SVM. MARS can transform variables, identify higher order interaction between variables which is a huge + vs. OLS.

For Question #1,

  • MARS can handle outliers very well.
  • MARS is distribution free, therefore no assumption. MARS is entirely data driven or in other words it learns from data.
  • When I model using MARS at work, I hardly ever worry about assumptions. However its always a good idea to look at residulas and see if the model fit is adequate and there is no pattern left.

For Question #2,

  • MARS is entirely a data driven method, therefore you might need adequate data to fit MARS. n = 33 is probably a small set for MARS. I would suggest fit both OLS and MARS and see which gives better fit.
$\endgroup$
  • $\begingroup$ Many thanks for all your precise responses. But, now I am quite annoyed because I am wondering when or why I should better use MARS regression than high-breakdown regression methods (robust regression like the least median of squares (LMS) or LTS ? Both of them (MARS and robust regression) are useful when there are outliers and when there are some deviations from the classical assumptions from the standard regression model. $\endgroup$ – varin sacha Dec 9 '13 at 20:38
  • $\begingroup$ @varinsacha I'm not familiar with eithe LMS or LTS. I use MARs when I suspect nonlinearity and higher order interaction. MARS is known to be superior to other regression methods and can easily interpretable. $\endgroup$ – forecaster Dec 9 '13 at 21:16
  • $\begingroup$ Many Thanks once more for your response. I use the R package "earth" to realize MARS. The big problem with MARS is the same as every nonparametric regression. Indeed nonparametric estimators are much harder to interpret. Are You ok ? Best $\endgroup$ – varin sacha Dec 10 '13 at 19:53
  • $\begingroup$ @varinsacha, one of the strengts of MARS is it is highly intrepretable vs. other techniques. $\endgroup$ – forecaster Dec 10 '13 at 20:34
  • $\begingroup$ Hi forecaster, Thanks a lot for all your comments. I will send another message because I have realized a MARS regression with R and I don't understand everything for the interpretation. $\endgroup$ – varin sacha Dec 14 '13 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.