What is the difference between k-holdout and k-fold cross validation? Can some one provide an example on what is k-holdout and k-fold cross validation.  
 A: Basically, it's explained on Wikipedia:

k-fold cross-validation. In k-fold cross-validation, the original
sample is randomly partitioned into k equal sized subsamples. Of the k
subsamples, a single subsample is retained as the validation data for
testing the model, and the remaining k − 1 subsamples are used as
training data. The cross-validation process is then repeated k times
(the folds), with each of the k subsamples used exactly once as the
validation data.
Holdout method. This can be considered the simplest variation of
k-fold cross-validation, although it does not cross-validate. We
randomly assign data points to two sets d0 and d1, usually called the
training set and the test set, respectively. The size of each of the
sets is arbitrary although typically the test set is smaller than the
training set. We then train on d0 and test on d1.

So, simply saying, k-fold creates many randomly chosen subsamples (d1,d2,d3,...), holdout creates one random subsample for validation set (d1).

Given a sample with 9 data points:
[1,2,3,4,5,6,7,8,9]

3-fold:
validation: [1,5,7], training: [2,3,4,6,8,9] 
validation: [2,3,6], training: [1,7,4,5,8,9] 
validation: [4,8,9], training: [2,3,1,6,4,7]

$\binom 93 = 84$ (possible) cases overall (for exhaustive case)
9-fold (leave-one-out gives 9 folds) :
validation: [2], training: [1,3,4,5,6,7,8,9]
validation: [3], training: [1,2,4,5,6,7,8,9]
validation: [5], training: ...
validation: [4], training: ...
validation: [6], training: ...
validation: [1], training: ...
validation: [7], training: ...
validation: [9], training: ...
validation: [8], training: ...

holdout (validation subsample size = 4):
validation: [1,2,7,8], training: [3,4,5,6,9]
(1 subsample overall)

Note that holdout uses only one single validation-training pair, that's why it doesn't really cross-validate as there is only one subsample.
Basically holdout procedure is same as for choosing a test-set. The only difference is that when you train several models (and several variations of hyper-parameteres) in assembly - you choose a model that performs best on the validation set (minimal validation error), but report test-set error because it's less optimistic.

P.S. There is also an important distinction between exhaustive and non-exhaustive cross-validation. There are 84 possible splits for 3-fold of 9 points, but only some small number of subsamples is used in non-exhaustive case, otherwise it would be a "Leave-p-out" (Leave-3-out) cross-validation (it validates all 84 subsamples)
