Why is it wrong to use non-nested cross-validation? Imagine I have a classifier with some parameter $a$. I want to perform parameter optimisation and model selection. Imagine that my grid for the values of $a$ has three values 1,5 and 10 and I want perform 10-fold cross-validation as follows:


*

*For each iteration, the dataset is partitioned into a 9:1 ratio of training and testdata, whereby at each iteration the two consists from other sampled datapoints of the full dataset (which is the description of 10-fold cross-validation)

*Now, each of these iterations is done three times: one time for each parameter value.
Hence I end up with 30 iterations (10-fold cv basically, but for each parameter once) and pick the parameter which on average performed best. I am aware that this chosen model has not YET been tested on new data. However during all the iterations, since test and training set have been disjunct and independent, it's performance has been evaluated on this unseen test set. So why is this approach incorrect (as in comparison with nested cv with an inner and outer loop)? 
 A: The approach is not incorrect per se; it depends on the purpose you are using it for. For parameter optimisation and model selection it is fine, though you should iterate the cross validation for a more stable selection.
An error would be to just use the performance estimates obtained for the best model as your expectation of future model performance. 
Some intuition on that:
Consider the extreme case that the real model performance $p_i$ is identical for all three possible hyperparameter values. Since you have a finite sample the observed model performance is
$$
\hat{p}_i= p_i + \epsilon_i
$$
Here $\epsilon_i$ is an error term. If you now select the model with the best $\hat{p}_i$ it is very likely that the corresponding $\epsilon_i$ is positive. If we assume that $\epsilon_i$ is symmetrical around zero, then the probability of at least one $\epsilon_i$ being posiitve is $\frac{7}{8}$ and obviously if one is positive we won't get a negative one by choosing the highest $\hat{p}_i$. This means you will overestimate the real performance of the model.
Now, in a nested cross validation what you do (I agree with @amoeba that what you do is not nested CV) is the inner or nested part of the cross validation. Out of it you get a single model which you apply to the training data of the outer CV loop (which is new to the model). 
This means you get a new error estimate with is independent from the one in the inner cross validation and therefore does not inherit the bias. Since you have just one model at this point you do not need to made a choice of models, which would introduce a new bias.
