Regressor predicting accuracy on previously seen data

Question

I am applying various regression algorithms, such as RandomForestRegressor, AdaBoostRegressor, KNeighborsRegressor ...etc. I fit these models on training data (only 1 feature for simplicity), and intentionally leaking the training data to observe the prediction accuracy. In other words, I am using the training data as testing data. As I understand, theoretically the prediction accuracy should be 100% as all the data have been seen before, but surprisingly I am getting completely different outcome (RMSE: 1.64).

df=pd.read_csv('myfile.csv', header = 0)
df2=df[['feature1','target']]
y = df2.pop('target').values
X = df2.values
knn = make_pipeline(StandardScaler(), KNeighborsRegressor(n_neighbors=4))
knn.fit(X, y)
df['KNN_predicted']=knn.predict(X)
ypre=knn.predict(X)
mse = mean_squared_error(y, ypre)
print("RMSE: ", mse**(1/2.0))

The same thing for RandomForestRegressor, AdaBoostRegressor, SGDRegressor, MLPRegressor and SVR.

Answer 1

To add to Stephan's comment - the fact that you are evaluating your model with the training data does not imply that the model is completely overfitted.

In your case, you are using only one feature. It is likely that more information is needed to predict your response variable perfectly.

In other words, your simple model probably suffers from high bias, and so is unable to predict the training data without error.