I am fairly new to the machine learning, and I have been going over all the great posts about cross-validation today and I have a question regarding PCA and cross-validation, I don't have enough points to comment on the [PCA and train/test split](http://stats.stackexchange.com/questions/55718/pca-and-the-train-test-split?newreg=cdaa370fba864f4dbf8d7f9e5a938ffa) so I thought maybe I should post a new question.

My understanding is the most correct procedure is the following which I saw in an earlier [post](http://stats.stackexchange.com/a/46251):


    for each fold:
        split data
        conduct PCA on the 90% used for training
        pick the number of components
        fit linear regression
        predict the 10% held out
    end

My main question is, if I want to do eigenfaces with PCA and SVM I would split up my set of images into my training and validation sets, and then apply the PCA to each new split in my cross-validation and optimization?  My confusion comes because I was following an example on [Scikit-learn](http://scikit-learn.org/stable/auto_examples/applications/face_recognition.html) where they divide the data, and then proceed to caluclate the PCA for the split data. Next, they run [GridSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html#sklearn.grid_search.GridSearchCV) which I understand is doing a cross-validation to tune the parameters. Does this tuning introduce a bias because it preprocessed the data or is it okay for some reason? I have attached the relevant sections of the example below.

    ###############################################################################
    # Split into a training set and a test set using a stratified k fold
    
    # split into a training and testing set
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.25)
    
    
    ###############################################################################
    # Compute a PCA (eigenfaces) on the face dataset (treated as unlabeled
    # dataset): unsupervised feature extraction / dimensionality reduction
    n_components = 150
    
    print("Extracting the top %d eigenfaces from %d faces"
          % (n_components, X_train.shape[0]))
    t0 = time()
    pca = RandomizedPCA(n_components=n_components, whiten=True).fit(X_train)
    print("done in %0.3fs" % (time() - t0))
    
    eigenfaces = pca.components_.reshape((n_components, h, w))
    
    print("Projecting the input data on the eigenfaces orthonormal basis")
    t0 = time()
    X_train_pca = pca.transform(X_train)
    X_test_pca = pca.transform(X_test)
    print("done in %0.3fs" % (time() - t0))
    
    
    ###############################################################################
    # Train a SVM classification model
    
    print("Fitting the classifier to the training set")
    t0 = time()
    param_grid = {'C': [1e3, 5e3, 1e4, 5e4, 1e5],
                  'gamma': [0.0001, 0.0005, 0.001, 0.005, 0.01, 0.1], }
    clf = GridSearchCV(SVC(kernel='rbf', class_weight='auto'), param_grid)
    clf = clf.fit(X_train_pca, y_train)
    print("done in %0.3fs" % (time() - t0))
    print("Best estimator found by grid search:")
    print(clf.best_estimator_)