What's the real purpose of cross validation? As for cross evaluation (CV), I have two questions to ask:
1) CV has nothing to do with parameter selection, but only model evaluation? Specifically, which model?
2) In k-fold CV, what's the final model to be retained? I understand that in each fold, the model is created and them discarded immediately?
I am new to ML and get confused about the purpose of CV.
 A: 
1) CV has nothing to do with parameter selection, but only model evaluation? 

Yes, CV is about model evaluation or verification (which is a sub-task of validation) or performance estimation or performance measurement. 
The link to parameter selection is that some strategies for parameter selection rely on performance estimates or measurements, and those can be obtained via cross validation.
So the relation is a rather weak (parameter selection can use other criteria instead of estimated generalization error, and even that can be estimated by other thechniques than CV) one-way type of relation. 

Specifically, which model?

Any model. One advantage of cross validation is that it works completely independent of the model as it treats the model as a black box that spits out predictions when applied to input data 

2) In k-fold CV, what's the final model to be retained? 

Typically, k-fold CV (or any other restampling validation technique) is used to get an approximation to the generalization performance (performance of predictions for unknown cases) of the model trained on all available cases. So none of the $k$ models is retained, but the results of the k-fold CV procedure are used as approximation/estimate for the performance of yet another model (the one obtained by using the same training procedure together with all cases).

I understand that in each fold, the model is created and them discarded immediately?

Typically yes, but sometimes these so-called surrogate models are retained for further inspection (e.g. wrt. stability).
They can also be used for model aggregation, but if you do this you'll have to keep in mind that the original CV performance estimate does not estimate predictive performance of the aggregated model. 
A: Perhaps I can help by explaining the reason cross-validation is used. 
Motivation: the situation for each model
If $\tilde{y}$ is future/unobserved data and $y$ is data you have, a scoring rule/function $S(p,\tilde{y})$ is a function that takes


*

*the distribution you're using to forecast $p$ (estimated from $y$) , and 

*a realized future/unobserved value $\tilde{y}$
and then gives you a real-valued number/score/utility. Higher is better, although this convention isn't always followed in the literature. This score is random because it's a transformation of a random future data point, so people are usually interested in its expected value. The expected value tells you the predictive capabilities of your model. We could write this as 
$$
E_f[S(p,\tilde{y})]
$$
where $f$ is the unknown true distribution of $\tilde{y}$. For example, a popular scoring rule is $S(p,\tilde{y}) = \log p(\tilde{y})$. You take the future data point, and plug it into your log-likelihood. 
Approximation
The trouble though is that the expected score is hard to get at because 


*

*you don't know the true distribution of $\tilde{y}$, and

*you don't want to wait around to get samples of this future data (if you did, you could the Monte-Carlo technique and take the average score to get something really close).


Also, you can't just plug in $y$ for $\tilde{y}$ because this will overstate your models' predictive capability. This is why all of this data splitting comes into play. 
So here's the point: cross-validation is a way to estimate this expected score. You repeatedly partition the data set into different training-set-test-set pairs (aka folds). For each training set, you estimate the model, predict, and then obtain the score by plugging the test data into the probabilistic prediction. This gives you one score for each fold. Then you take the average of these (or the sum, depending on what reference you're using).
A few things about CV:


*

*The partitions are nonrandom, test sets are disjoint

*for each split/estimation/prediction, we never use a data point twice

*for each split/estimation/prediction, we lose parameter estimation accuracy because each training set is smaller than the full set

*however, we get to average over many prediction scores, which reduces variance

*the estimator is still biased, so people may or may not try to correct for this

*it can be computationally brutal to calculate for some models

*the logo of this site illustrates this splitting procedure!


Many models
And yes, you are right, after this whole thing is done for multiple models, you can see which model has the the best (estimated) predictive capability. You can pick that model, and then estimate it using the full data set. Before you implement this in practice, you should know that blindly doing CV for a large collection of models will rarely help you pick the correct one, but that's a question for another thread.
