Steps after cross-validation in Linear Regression

Question

After making some variable transformations, I chose this model because I works well with the assumptions and gives the largest R-Squared and the smallest Overall MS in cross-validation between a set of linear models.

> gn<- lm(NA.~ I(PC^0.25) + I(((PI)^2)),data=DSET)
> summary(gn)

Call:
lm(formula = NA. ~ I(PC^0.25) + I(((PI)^2)), data = DSET)

Residuals:
    Min      1Q  Median      3Q     Max 
-425.22  -87.46   -2.30   79.11  396.14 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -2.047e+03  1.094e+02  -18.71  < 2e-16 ***
I(PC^0.25)   1.206e+03  4.231e+01   28.52  < 2e-16 ***
I(((PI)^2)) -5.242e-02  1.233e-02   -4.25 2.81e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 127.2 on 319 degrees of freedom
Multiple R-squared:  0.7475,    Adjusted R-squared:  0.746 
F-statistic: 472.3 on 2 and 319 DF,  p-value: < 2.2e-16

The output of the cross-validation is the following:

> a<-CVlm(df=DSET,form.lm = gn ,m=5)
Analysis of Variance Table

Response: NA.
             Df   Sum Sq  Mean Sq F value  Pr(>F)    
I(PC^0.25)    1 14979750 14979750   926.5 < 2e-16 ***
I(((PI)^2))   1   292051   292051    18.1 2.8e-05 ***
Residuals   319  5157489    16168                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1



fold 1 
Observations in test set: 64 


Sum of squares = 1119289    Mean square = 17489    n = 64 

fold 2 
Observations in test set: 65 

Sum of squares = 1072687    Mean square = 16503    n = 65 

fold 3 
Observations in test set: 65 

Sum of squares = 865598    Mean square = 13317    n = 65 

fold 4 
Observations in test set: 64 


Sum of squares = 1178545    Mean square = 18415    n = 64 

fold 5 
Observations in test set: 64 

Sum of squares = 1042335    Mean square = 16286    n = 64 

Overall (Sum over all 64 folds) 
   ms 
16393 


> 

So, my question is, Which steps do you recommend to me after the cross-validation? Should I choose the training set applied to the fold 3(it gives the small mean square error)? Is there a way to improve the coefficients after the cross-validation?

Answer 1

The output seems to list the cross-validation results from a single model. I assume what you did was produce analogous results for the models you were selecting between and are only displaying the results from the best model. Having decided that this is the best model, the standard practice is then to fit that model to the full dataset outside of cross-validation, as it seems you did at the top of your question. Then you can just report the coefficients and standard errors from this model. Note that these standard errors are optimistic (smaller than they should be) in that they don't take into account the fact that a different sample of data might have lead to you selecting a different "best" model.