I'm training a Random Forest Regressor and I'm evaluating the performances. I have an MSE of 1116 on training and 7850 on the test set, suggesting me overfitting.
I would like to understand how to optimize the algorithm quality in generalization starting from cross-validation technique.
I did:
from sklearn.ensemble import RandomForestRegressor
from sklearn import model_selection
from sklearn import metrics
rfcv=RandomForestRegressor()
cv = model_selection.KFold(n_splits=8)
for (train, test), i in zip(cv.split(X_train, y_train), range(8)):
rfcv.fit(X_train.iloc[train], y_train.iloc[train])
y_pred = rf.predict(X_test)
print (metrics.mean_squared_error(y_test, y_pred))
model=RandomForestRegressor()
accuracy = cross_val_score(model, df_final_X_hot, df_final_y, scoring='r2', cv = 10)
print(accuracy)
Now, I would like to understand how to use the results. what indication does cross validation give me? the algorithm that I have to use for my predictions is the one obtained from this cross validation?