Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Many textbooks and papers said that intercept should not be suppressed. Recently, I used a training dataset to build a linear regression model with or without an intercept. I was surprised to find that the model without an intercept predicts better than that with an intercept in terms of rmse in an independent validation dataset. Is the prediction accuracy one of the reasons that I should use zero-intercept models?

share|improve this question
    
How big were the training and validation sample sizes? Maybe the model without an intercept was better just by chance. –  mark999 Jan 26 '12 at 4:15
    
The training sample size was 289 whereas the validation sample size was 406. By the way, how to determine the best training and validation sample sizes? –  Jinn-Yuh Guh Jan 26 '12 at 5:22

4 Answers 4

up vote 2 down vote accepted

Look carefully at how the rmse or other statistic is computed when comparing no-intercept models to intercept models. Sometimes the assumptions and calculations are different between the 2 models and one may fit worse, but look better because it is being divided by something much larger.

Without a reproducible example it is difficult to tell what may be contributing.

share|improve this answer
    
Rmse was calculated according to the formula (used to compare differences between two things that may vary, neither of which is accepted as the "standard") given in: en.wikipedia.org/wiki/Root-mean-square_deviation So the assumptions and calculations are the same between the 2 model-derived estimators. –  Jinn-Yuh Guh Jan 27 '12 at 2:38
    
How similar are your training and validation sets? You might consider doing 10-fold cross validation: split the data into 10 equal (or as equal as you can get) pieces randomly, then use 9 of them to train a model and the 10th as the validation piece, then repeate with each of the other 9 pieces being the validation set. Then repeate that whole process (starting with a new random split) 10 times or so. –  Greg Snow Jan 27 '12 at 15:49
    
Y variable and X variables were different (P=0.01) between the training and validation sets. However, rmse was still lower with the model with no intercept when I used the R package MatchIt to match the training and validation sets. In contrast, rmse became similar when I used the DAAG package (cv.lm) to 10-fold cross-validate the combined dataset. Does this mean that 10-fold cross-validation is better than a simple training set and a validation set? –  Jinn-Yuh Guh Jan 28 '12 at 10:03
1  
Yes the cross-validation tends to be better. Look to see if there are other ways that your model is over specified, it is very unusual for a non-intercept model to fit better. –  Greg Snow Jan 28 '12 at 23:42
    
In "To Explain or to Predict?" projecteuclid.org/… Professor Galit Shmueli said that sometimes a less true model can predict better than a truer model. I think this may be one of the reasons of this case. –  Jinn-Yuh Guh Feb 3 '12 at 15:02

I don't think you should choose models simply because they work better in a particular sample, although it is good that you used a training and validation sample.

Rather, look at what the models say about your situation. In some cases a zero-intercept model makes sense. If the DV ought to be 0 when all the IVs are 0, then use a zero-intercept model. Otherwise, don't.

Substantive knowledge should guide statistics, not the other way around

share|improve this answer
1  
The reason given in your second paragraph, while intuitive, is often not a strong enough one to suppress the intercept in many such situations. This point is addressed more fully in a couple other questions on this site. –  cardinal Jan 26 '12 at 13:51
4  
In method (or instrument) comparison studies (e.g. the comparison of oximeter A and oximeter B), the DV (oxygen level) ought to be 0 when all the IVs (oxygen levels) are 0. However, the intercept should not be ignored if I want to calibrate (or exchange) oximeter A with oximter B. –  Jinn-Yuh Guh Jan 26 '12 at 14:28

This would be understandable if the intercept you obtained was merely noise --not sig. different from zero. (Am I right that the standardized regression coefficients were nearly the same in both models?) If so I don't think you should generalize from this example. When intercepts are sig. and substantial, they add something meaningful to predictive accuracy.

share|improve this answer
    
1. The standardized regression coefficients were not the same (0.91 and 1.02) for the model with and without an intercept). 2. The intercept was 9.5 (s.e. 1.7, p<0.001). 3. As far as I know, many papers suggest not to suppress the intercept even if the intercept was not significant from zero. –  Jinn-Yuh Guh Jan 26 '12 at 2:51
    
If there are dummy variables in the regression, doesn't the intercept represent the value if all dummies are 0-coded for that observation? Not sure if this applies here. –  Michelle Jan 26 '12 at 9:07
    
No, there were no dummy variables. –  Jinn-Yuh Guh Jan 26 '12 at 14:00

A no intercept model may make sense if two conditions are met. First, there should be a reasonable subject matter knowledge expectation for the intercept to be zero. Second, there should be a reasonable subject matter knowledge expection for the regression line to remain a straight line as you approach zero. Even if both conditions are satisfied, it is wise to run an analysis with an intercept term and verify that the intercept is not significantly different from zero.

(I am assuming that you are talking about a continuous Y and a continuous X.)

share|improve this answer
    
+1 Very well put. –  whuber Feb 1 '12 at 17:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.