Skip to main content
formatted; light editing; removed thanks
Source Link
gung - Reinstate Monica
  • 147.5k
  • 89
  • 406
  • 717

Scikit Learn's page on Model Selectionpage on Model Selection mentions the use of nested cross-validation at:

http://scikit-learn.org/stable/tutorial/statistical_inference/model_selection.html

>>> clf = GridSearchCV(estimator=svc, param_grid=dict(gamma=gammas),
>>> clf = GridSearchCV(estimator=svc, param_grid=dict(gamma=gammas),
...                    n_jobs=-1)
>>> cross_validation.cross_val_score(clf, X_digits, y_digits)

"Two cross-validation loops are performed in parallel: one by the GridSearchCV estimator to set gamma and the other one by cross_val_score to measure the prediction performance of the estimator. The resulting scores are unbiased estimates of the prediction score on new data."

>>> cross_validation.cross_val_score(clf, X_digits, y_digits)

Two cross-validation loops are performed in parallel: one by the GridSearchCV estimator to set gamma and the other one by cross_val_score to measure the prediction performance of the estimator. The resulting scores are unbiased estimates of the prediction score on new data.

From what I understand, clf.fitclf.fit will use cross-validation natively to determine the best gamma. In that case, why would we need to use nested cv as given above.? The note mentions that nested cv produces "unbiased estimates" of the prediction score. Isn't that also the case with clf.fit clf.fit?

Also, I was unable to get the clf best estimates from the cross_validation.cross_val_score(clf, X_digits, y_digits)cross_validation.cross_val_score(clf, X_digits, y_digits) procedure. Could you please advise how that can be done.

Thanks in advance.?

Scikit Learn's page on Model Selection mentions the use of nested cross-validation at

http://scikit-learn.org/stable/tutorial/statistical_inference/model_selection.html

>>> clf = GridSearchCV(estimator=svc, param_grid=dict(gamma=gammas),
...                    n_jobs=-1)
>>> cross_validation.cross_val_score(clf, X_digits, y_digits)

"Two cross-validation loops are performed in parallel: one by the GridSearchCV estimator to set gamma and the other one by cross_val_score to measure the prediction performance of the estimator. The resulting scores are unbiased estimates of the prediction score on new data."

From what I understand, clf.fit will use cross-validation natively to determine the best gamma. In that case, why would we need to use nested cv as given above. The note mentions that nested cv produces "unbiased estimates" of the prediction score. Isn't that also the case with clf.fit ?

Also, I was unable to get the clf best estimates from the cross_validation.cross_val_score(clf, X_digits, y_digits) procedure. Could you please advise how that can be done.

Thanks in advance.

Scikit Learn's page on Model Selection mentions the use of nested cross-validation:

>>> clf = GridSearchCV(estimator=svc, param_grid=dict(gamma=gammas),
  ...                    n_jobs=-1)  
>>> cross_validation.cross_val_score(clf, X_digits, y_digits)

Two cross-validation loops are performed in parallel: one by the GridSearchCV estimator to set gamma and the other one by cross_val_score to measure the prediction performance of the estimator. The resulting scores are unbiased estimates of the prediction score on new data.

From what I understand, clf.fit will use cross-validation natively to determine the best gamma. In that case, why would we need to use nested cv as given above? The note mentions that nested cv produces "unbiased estimates" of the prediction score. Isn't that also the case with clf.fit?

Also, I was unable to get the clf best estimates from the cross_validation.cross_val_score(clf, X_digits, y_digits) procedure. Could you please advise how that can be done?

Source Link
xbsd
  • 805
  • 1
  • 9
  • 18

Use of nested cross-validation

Scikit Learn's page on Model Selection mentions the use of nested cross-validation at

http://scikit-learn.org/stable/tutorial/statistical_inference/model_selection.html

>>> clf = GridSearchCV(estimator=svc, param_grid=dict(gamma=gammas),
...                    n_jobs=-1)
>>> cross_validation.cross_val_score(clf, X_digits, y_digits)

"Two cross-validation loops are performed in parallel: one by the GridSearchCV estimator to set gamma and the other one by cross_val_score to measure the prediction performance of the estimator. The resulting scores are unbiased estimates of the prediction score on new data."

From what I understand, clf.fit will use cross-validation natively to determine the best gamma. In that case, why would we need to use nested cv as given above. The note mentions that nested cv produces "unbiased estimates" of the prediction score. Isn't that also the case with clf.fit ?

Also, I was unable to get the clf best estimates from the cross_validation.cross_val_score(clf, X_digits, y_digits) procedure. Could you please advise how that can be done.

Thanks in advance.