I am applying various regression algorithms, such as RandomForestRegressor, AdaBoostRegressor, KNeighborsRegressor ...etc. I fit these models on training data (only 1 feature for simplicity), and intentionally leaking the training data to observe the prediction accuracy. In other words, I am using the training data as testing data. As I understand, theoretically the prediction accuracy should be 100% as all the data have been seen before, but surprisingly I am getting completely different outcome (RMSE: 1.64).

df=pd.read_csv('myfile.csv', header = 0)
y = df2.pop('target').values
X = df2.values
knn = make_pipeline(StandardScaler(), KNeighborsRegressor(n_neighbors=4))
knn.fit(X, y)
mse = mean_squared_error(y, ypre)
print("RMSE: ", mse**(1/2.0))

The same thing for RandomForestRegressor, AdaBoostRegressor, SGDRegressor, MLPRegressor and SVR.

  • 1
    $\begingroup$ There is presumably residual variation, as there always is in the real world. Not everything can be modeled and predicted. $\endgroup$ Nov 14, 2022 at 16:26

1 Answer 1


To add to Stephan's comment - the fact that you are evaluating your model with the training data does not imply that the model is completely overfitted.

In your case, you are using only one feature. It is likely that more information is needed to predict your response variable perfectly.

In other words, your simple model probably suffers from high bias, and so is unable to predict the training data without error.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.