Why should we use cross_val_predict instead of just normally predicting our instances using all the training set?

Question

Aren't we using less data to train our model when we are using cross_val_predict? Say I have the following code;

sgd_clf = SGDClassifier(random_state=42)
sgd_clf.fit(X_train, y_train)

y_train_pred_fullData = sgd_clf.predict(X_train)

y_train_pred_cv = cross_val_predict(sgd_clf, X_train, y_train, cv=3)

Based on my understanding how cross_val_predict works (with cv=3) is that it divides the training set into three equal chunks and it trains on the 2nd and 3rd chunk to predict the labels of instances in the 1st one third of the instances in the training set, and uses the 1st and 3rd chunk to predict the labels of the 2nd one third of the instances, and lastly it uses the 1st and 2nd one third of the training data to predict the labels of the 3rd chunk. Please correct me if I am wrong in my understanding.

Given the above, wouldn't it be better for me to use y_train_pred_fullData rather than y_train_pred_cv because I am using more data to train my model to make predictions?

If I would prefer y_train_pred_fullData over y_trian_pred_cv under the conditions of using more data to train on, under what conditions would I choose to use y_train_pred_cv?

Kind regards

Answer 1

Yes, we don't use cross_val_predict for that. You use cross validation to compare performance of different estimators and select the best one. You then fit the best estimator on the full data. Default options for model comparison are cross_validate (when you have multiple metrics to check) and cross_val_score (when you have only one metric).

For usages of cross_val_predict, the documentation states:

The function cross_val_predict is appropriate for:

Visualization of predictions obtained from different models.

Model blending: When predictions of one supervised estimator are used to train another estimator in ensemble methods.