1
$\begingroup$

I'm learning linear regression with the Carseats data set. I went through the data, cleaned it, encoded the dummy variables and checked for collinearity.

The dataset has 400 observations on Carseat sales with information on Sales, CompPrice, Income, Advertising, Population, Price, Age and Education.

I'm having trouble figuring out which model is the best for predicting Sales.

Initialize the dataset

library('ISLR')
data(Carseats)
attach(Carseats)

Summary on the datasets:

Original dataset: Carseats 
Scaled Predictors: Carseats_S
Scaled Predictors and Encoded Categoricals: Carseats_SD 

Here are my models

fit1 <- lm(Sales~., data=Carseats_SD)
fit2 <- lm(Sales~., data=Carseats)
fit3 <- lm(Sales~., data=Carseats_N)
fit4 <- lm(Sales~., data=Carseats_NSD)
fit5 <- lm(Sales~.+(Price*CompPrice), data=Carseats_SD)
fit6 <- lm(Sales~Price+Income+Advertising+Age, data=Carseats_SD)
fit7 <- lm(Sales~Price+Income+Advertising+Age+ShelveLoc_Bad+ShelveLoc_Good+ShelveLoc_Medium, data=Carseats_SD)
fit8 <- lm(Sales~.-CompPrice, data=Carseats_SD)
fit9 <- lm(Sales~.-Age-Education-Income-age_segment_old-age_segment_young-age_segment_middle_aged-Advertising, data=Carseats_NSD)
fit10 <- lm(Sales~Price+ShelveLoc_Good, data=Carseats_SD)
fit11 <- lm(Sales~Price+ShelveLoc_Good+Education+Advertising, data=Carseats_SD)
fit12 <- lm(Sales~.-ShelveLoc_Bad-ShelveLoc_Medium, data=Carseats_SD)

And here's the output with diagnostics information

   id LOOCV_MSE Validation_MSE 5Fold_MSE 10Fold_MSE  FStat Adjusted_R2    RSS     AIC     BIC
1   1      0.13           0.00      0.14       0.13 243.37        0.87  50.51  333.42  385.31
2   2      1.07           0.01      1.07       1.06 243.37        0.87 402.83 1163.97 1215.86
3   3      1.10           0.00      1.10       1.11 140.21        0.87 397.25 1174.39 1258.21
4   4      0.14           0.00      0.14       0.14 140.21        0.87  49.81  343.84  427.66
5   5      0.13           0.00      0.14       0.14 222.99        0.87  50.41  334.67  390.55
6   9      0.18           0.00      0.18       0.18 129.78        0.83  65.74  446.84  514.69
7  12      0.22           0.00      0.22       0.22 150.73        0.79  81.85  524.52  572.41
8   7      0.30           0.00      0.30       0.30 158.33        0.70 116.76  658.61  690.54
9   8      0.31           0.00      0.31       0.31  95.25        0.70 115.70  662.95  710.85
10 11      0.47           0.00      0.47       0.47 115.42        0.53 183.97  836.48  860.43
11 10      0.54           0.00      0.54       0.54 175.96        0.47 211.51  888.27  904.24
12  6      0.64           0.00      0.66       0.64  58.21        0.36 251.03  960.79  984.74

For starters, I made these models based on testing interaction terms, using different adjustments on the dataset and trying variables based on best subset selection.

The diagnostic plots for the models look good. They don't indicate heteroscedasticity, correlated error terms, outliers or non-linear relationships.

How do you choose which model is best?

$\endgroup$
  • $\begingroup$ What is your objective? To find the model which best fits the data (for interpretation) or the best model for prediction? $\endgroup$ – user2974951 Jan 18 at 8:54
  • $\begingroup$ primarily prediction $\endgroup$ – Sebastian Jan 18 at 17:20
0
$\begingroup$

If prediction is the primary objective then your primary goal is to minimize error. Since this is a regression problem and you do not have a test set, then the best alternative is CV (LOOCV) using a standard measure such as MSE. Note: LOOCV asymptotically converges to the log likelihood score, which you can easily obtain from a linear model using the anova function. In your case, the best models (which minimize MSE LOOCV) are either 1, 4 or 5. Usually you would choose the simplest model when you have many options.

You do not have to concern yourself with diagnostics if your primary objective is prediction, but it would be important if your objective is interpretation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.