Summary: To my knowledge, there is no ready-to-use formula yet for the uncertainty of cross validation results (yet).
You can calculate a standard error across fold-wise estimates as in the linked question.
But the standard error of fold-wise estimates is not what we usually need to draw further conclusions. In the correct answer to the wrong question sense.
In particular, it does not* give a good approximation to answers about the uncertainty of the CV results wrt. the predictive ability of the model at hand (trained by the algorithm on the full data set at hand). (And even less about the training algorithm.)
- or maybe only under particular circumstances (stable models, many cases). Of which I can already say: in my daily work they are not usually met.
Model agnostic
- Cross validation in all its flavors like other resampling techniques for estimating generalization error test in a model-agnostic way (treats the model as black box), so whatever formula you get will automatically be model agnostic.
- Loss functions make a differnce, though. (E.g. for proportions of tested cases, one can derive boundaries even before the test starts - which is not possible e.g. for regression-type loss functions)
Nesting
I consider the optimization done in the inner nesting loop part of the training of the final model.
From that point of view, it may be interesting to see how well the optimizer works, but the outer CV loop can and should be treated separately: as a (non-nested) CV experiment of a black box model. That this model internally uses optimization with another CV loop is not relevant for the treatment of the outer (verification) CV.
It will thus be sufficient to know how to deal with not-nested CV.
IMHO the difficulty is connecting standard deviations observed during cross validation to uncertainty of the final cross validation result - which is usually the reason why we want to know standard error.
Variance in cross validation experiments
Cross validation has at least 2 different sources of variance uncertainty:
instability of the surrogate models
case-to-case variance => variance due to the limited, finite number of tested cases
(When talking about the general ability of the algorithm under consideration to solve problems of the given type with data sets like the one at hand - as opposed to estimating generalization error of the model trained by the algorithm on the data at hand - further sources of uncertainty come in)
These variances are both "mixed" in the variance observed across CV folds, or in pooled predictions.
Yet when looking at the uncertainty for the cross validation result, both are needed in different ways:
Model instability: since in the end we use the CV results to characterize single models, model instability variance itself (if you like, divided by n = 1 model) is the standard error estimate for this source of variance error.
Repeated CV allows better estimation of this variance component.
Case-to-case variance: since we do CV in order to test all $n$ cases we have, and all those single tests are pooled into the final result: the standard error for this component has division by the total number of independent cases tested.
Repeated CV does not increase the $n$ here.
