# using MARS regression on data with outliers - R

In some previous asked questions, I was told to not delete the outliers, because they contain valuable information.

After testing different regression, I came to the conclusion that until now, the MARS regression delivers the "best responses".

I know that MARS is very robust and there is no a priori knowledge about the data distribution needed.

But there are some question which I have about the parameters.

I'm using the earth function implemented in R

data set: file

So I've got 5 variables, price, livingArea, area, discrete, dummy and I'm trying to explain price using the other ones.

as you can see, there are some outliers and a log doesn't really solve the problem. Due to the fact that area can be null, a log won't be a good transformation idea.

what I do:

Because the answers from other questions suggested to use the raw data, I'm running now the regression through my data without doing any changes to it.

so my regression formula looks like this:

earth(price ~ ., data = data[,-1], weights = weights, penalty = -1)


I'm setting penalty = -1 because I saw that doing this, the method defines more knots and also the results look better.

Also I tried to define the variables discrete and dummy as factors and use them as follows in the regression:

1. independent
2. livingArea * discrete or livingArea : discrete and dummy as independent
3. the same as at 3. but changing discrete with dummy
4. livingArea * discrete * dummy

I must say that I didn't expect, that a regression with this variables as factors, will return such "bad" results.

what I want:

I want to use the model in order to predict the value of new data.

    livingArea area discrete dummy
1         87    0        7    0.5


The prediction of this observation should be ~ 330000, but with what I'm doing now, I ain't coming not even close to this value.

I think that, having more knots increases the precision of the result.

questions:

• I don't really understand the parameters
• I created my model with different values for the pmethod, but the result was always the same. what's the point in choosing a method when the result will be the same?
• how could I determine if I have to set/change the values of different parameters like thresh, minspan, nk, etc.

This isn't what you ask for, but I think it's pertinent to analysing the data and it won't all fit easily into comments.

1. What does area mean if some values are 0? As a matter of interest I found that area and square root of area don't help much any way. That matches the evidence of the graphs in the question.

2. The variable dummy is not (0, 1), so I don't know how to think about it, but it doesn't help much either.

Following earlier suggestions I fitted a generalised linear model with log link. The choice of family seems fairly unimportant, but I haven't explored much. These results are from Stata. Two predictors seem strongly confirmed, but the overall fit is not strong.

. glm price log_livingarea discrete, link(log)

Iteration 0:   log likelihood = -3093.2279
Iteration 1:   log likelihood = -3078.9355
Iteration 2:   log likelihood = -3077.8055
Iteration 3:   log likelihood = -3077.8052
Iteration 4:   log likelihood = -3077.8052

Generalized linear models                         No. of obs      =        238
Optimization     : ML                             Residual df     =        235
Scale parameter =   1.01e+10
Deviance         =  2.38043e+12                   (1/df) Deviance =   1.01e+10
Pearson          =  2.38043e+12                   (1/df) Pearson  =   1.01e+10

Variance function: V(u) = 1                       [Gaussian]
Link function    : g(u) = ln(u)                   [Log]

AIC             =   25.88912
Log likelihood   = -3077.805214                   BIC             =   2.38e+12

--------------------------------------------------------------------------------
|                 OIM
price |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
log_livingarea |   .6915059   .0888826     7.78   0.000     .5172992    .8657126
discrete |   .1974758   .0247692     7.97   0.000      .148929    .2460226
_cons |   7.811162   .4801921    16.27   0.000     6.870003    8.752321
--------------------------------------------------------------------------------


I am not surprised that the model is not a spectacularly good fit. There are many possible predictors of house prices that are not in your data.

• 1. area is the lot area. a house will always have a lot area (garden, yard, etc.) but an apartment will not always have a lot area. Of course that I'm predicting apartments out of apartment data and house out of houses data. 2. yeah I know that dummy is between (0,1) but I just hadn't a better idea for naming it. The thing is that I had a lot of variables and after working with 12 predictors for ~7-8 months, I saw that combining this predictors and reducing the dimension of the data works fine. Maybe this is not a good statistical approach, but it works.
– Paul
Nov 10 '15 at 18:21
• the variable discrete is build out of 2 discrete variables and the variable dummy is build out of 4 dummy variables. sum(dummies)/4 and that's why it not 0 or 1.
– Paul
Nov 10 '15 at 18:23
• I must say that in 85% of the I have a nice data with a few outliers and a robust regression can handle it perfect, but in some cases (like this one) the data is terrible and also is the result.
– Paul
Nov 10 '15 at 18:24
• Thanks for the detail. As a matter of strategy, I would disentangle the discrete and dummy variables all the way back to the originals. Fit with all predictors, then slim down the model. The problem for anyone else with what you have done is that they cannot interpret those variables, which are just arbitrary composites. Nov 10 '15 at 18:25
• Messy data are just part of the job: you do the best you can and move on. Nov 10 '15 at 18:25