20-fold cross validation of a dataset composed of1000 observations? I have a dataset made up of 1000 observations and I want to split that data and use it for estimation of parameters in a way that maximum of 50 observations form one fold and my estimation process covers the whole data to estimate parameters, but only 50 observations at a time. Should I use 20-fold or I can adopt any other better method? Please note that I am new to machine learning and data mining. Please guide, thank you. 
 A: Using $k$-folds with $k = 20$ will yield subsamples of $n = \frac{1000}{20} = 50$; your estimator will then be trained using $k - 1 = 19$ of these, i.e. on a sample of $n=950$, and tested on the remaining subset. This is repeated so that each subsample is used for testing, giving error estimates which are typically averaged to estimate out-of-sample error. Each datum will be used once and only once, which the $k$-fold section of the Wikipedia article on cross-validation explains nicely [emphasis mine]:

The cross-validation process is then repeated k times (the folds), with each of the k subsamples used exactly once as the validation data. The k results from the folds can then be averaged (or otherwise combined) to produce a single estimation. The advantage of this method over repeated random sub-sampling (see below) is that all observations are used for both training and validation, and each observation is used for validation exactly once.

In other words, $k$-folds will use successive subsamples of $n = 50$ to estimate out-of-sample error; on each iteration, the parameters are estimated using the remaining $950$.
