# Cross Validation benefits LOOCV v.s K-Fold

I understand Cross Validation is used to parameter tuning and finding the machine learning model that will generalize well on the test data/

Leave one out cross validation: One data point is left out (validation) and trained on rest of the data points. This is done iteratively and then results are averaged out. But how do we find the best model when the results are average out ?

K-Fold we say K = 5 , 4 fold data is trained on and 1 fold data is validated and the results again are average out. Again how do we know which is the best model here ? Or do we choose a hyper parameter and do K-Fold here to see best hyper parameter in the model ?

Lastly, what is the difference between grid search CV and K-Fold ?

Cross validation is used to either compare different models or to tune the parameter. For the case of the parameter tuning, you can consider the models with different parameters to be unique models that you want to test. For both cases, the $MSE_i$ are averaged out for each model, and then compared to $CV$ parameter in different models. These $CV$ parameters cannot be used in absolute sense, they only make sense when compared to $CV$ parameters from the same data, same kind of cross validation, and different model (think of this like you would think of AIC and BIC). Once you have the $CV$ parameters from different models, you can choose the appropriate model (or parameter) by picking the smallest number (smaller the average $MSE$ the better).
• @Sheldon Lets assume we have a data with 12 potential predictors. First we try linear regression with 8 and 10 and 12 predictors. For linear regression no parameter tuning is needed because of the nature of the model, so we can either use K-fold or LOOCV to find the best model out of those three. The best model here is the one with the lowest $CV=\sum_i MSE_i$. Now imagine we try a Ridge regression. Parameter tuning is a common practice for Ridge regression, and here is when you need to tune your $\lambda$. You can again choose your LOOCV or K-fold and do a grid search for $\lambda$ and pick – V. Aslanyan Aug 27 '18 at 5:17
• parameter, fit a model, and perform a 5 fold crossvalidation. it will calculate a CV number for each of them and automatically choose a $\lambda$ for which this CV is the minimum. For better understanding of grid search and cross validation please have a look here – V. Aslanyan Aug 27 '18 at 5:38