I'm training a Random Forest Regressor and I'm evaluating the performances. I have an MSE of 1116 on training and 7850 on the test set, suggesting me overfitting.
I would like to understand how to optimize the algorithm quality in generalization starting from cross-validation technique.
from sklearn.ensemble import RandomForestRegressor from sklearn import model_selection from sklearn import metrics rfcv=RandomForestRegressor() cv = model_selection.KFold(n_splits=8) for (train, test), i in zip(cv.split(X_train, y_train), range(8)): rfcv.fit(X_train.iloc[train], y_train.iloc[train]) y_pred = rf.predict(X_test) print (metrics.mean_squared_error(y_test, y_pred)) model=RandomForestRegressor() accuracy = cross_val_score(model, df_final_X_hot, df_final_y, scoring='r2', cv = 10) print(accuracy)
Now, I would like to understand how to use the results. what indication does cross validation give me? the algorithm that I have to use for my predictions is the one obtained from this cross validation?