0
$\begingroup$

I am trying to find a good model for my data to predict new values. My idea was to combine a MARS and a CART model (averaging the outcomes of both models). This method fits the Training data well. Now I want to get some statistics which show how good this combined model is concerning prediction.

How can I estimate the predictive power in this case? Is cross-validation useful?

$\endgroup$
0
$\begingroup$

Boosting up till date is the most powerful way of combining CARTs. If by some way you think a model is only an estimate and should have a standard error, then model averaging is a good idea.

Note that you can always fit the training data as much as you want (over-fitting). And you will never judge a model on how it fits the training.

Therefore cross-validation bootstrapping and other out sample techniques are the only way to access the predictive performance of a model.

The performance measure use in assessing a model also plays an important rule. Predictive power is not often used, as power usually refers to hypothesis testing. Predictive performance is a better term to use.

$\endgroup$
1
  • $\begingroup$ Thanks for your answer. I want to try the cross-validation procedure. But I have just the basis functions of the MARS and the decision tree of the CART. I really dont know how to implement the Cross Validation for the average model (I need to know how the basis functions and the tree look like when I change the training data?). Could you give me a hint? $\endgroup$
    – R_FF92
    Oct 13 '15 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.