Idea and intuition behind two layer cross-validation I have seen people using two layer crossvalidation to evaluate their models and even in stats.stackexchange I've seen multiple posts where either two-layer crossvalidation was used or it was suggested. I haven't quite properly understood how and why two layer crossvalidation works or is useful. So can someone please elaborate on these two points:


*

*How does two-layer crossvalidation work and how it is used?

*When should it be used over standard crossvalidation and why?

 A: I guess that you are talking about the nested cross-validation (CV).
The whole idea of CV is to identify the optimal hyper-parameters for any given task. When you apply the usual 'single-layer' CV, you find the hyper-parameters that optimize the average performance of the CV folds. However, this may end up overfitting the CV loop itself since you iteratively use the same folds to optimize on.
If you want to calculate the expected performance of the CV-optimized hyper-parameters you could either use a test set that the model has never seen (with the risk of having an unstable test set score if the test set is not very large) or you can use the nested CV.
Below is some pseudocode:
for outer_train_set, outer_valid_set in train_set(K_fold):
    for inner_train_set, inner_valid_set in outer_train_set(K_fold):
        Train model and find the optimal hyper-parameters 
    Train model using the optimal hyper-parameters found on the inner loop
    Evaluate performance on outer_valid_set 

This way, you can get a proper estimation of the CV score that is not overfitted to the CV loop. Then, you'll have to apply the 'single-layer' CV (only the outer fold) in order to get the optimized hyper-parameters.
In short, you can use the nested-CV in order to get a more correct estimation of the CV performance. Sometimes you have to use nested-CV to optimize some hyper-parameters (see this link). In general, nested-CV is a better way for getting unbiased performance estimates but is not used in all projects due to much higher computational needs.
