I am new to Machine Learning, as part of my learning, I created a Linear Regression Model using two attributes for Boston Housing Price dataset. I am having doubts calculating : MSE and R-Square. So, if I train & split the data and calculate the R-square for Test or Training data and then if I use the k-fold cross validation, there is a significant difference in the average value. (0.11 vs 0.63 / 0.57)
Now, I understand that this is not the perfect model. But as I am beginner, I just wanted to understand if this kind of difference between values is normal and based on average cross validation score, we say this model is not the best fit irrspective of test/train r-square value? How would you interpret this difference?
import numpy as np
import pandas as pd
from sklearn.datasets import load_boston
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
from sklearn.model_selection import cross_val_score
boston = load_boston()
df = pd.DataFrame(boston.data)
df.columns = boston.feature_names
df['MEDV'] = boston.target
y_data = df['MEDV']
x_data = df.drop('MEDV',axis=1)
x_data.head()
x_train, x_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.15, random_state=10)
# Model using 'RM' & 'LSTAT'
lr = LinearRegression()
lr.fit(x_train[['RM','LSTAT']],y_train)
# Prediction using Training Data
Y_trainpr = lr.predict(x_train[['RM','LSTAT']])
# Prediction using Test Data
Y_testpr = lr.predict(x_test[['RM','LSTAT']])
# Model evaluation using Test Data
mse1_test = mean_squared_error(y_test,Y_testpr)
print("Mean square error is",mse1_test)
print("R-Square value using test data is", lr.score(x_test[['RM','LSTAT']],y_test))
print("\n")
# Model evaluation using Training Data
mse1_train = mean_squared_error(y_train,Y_trainpr)
print("Mean square error is",mse1_train)
print("R-Square value using training data is", lr.score(x_train[['RM','LSTAT']],y_train))
print("\n")
Rcross = cross_val_score(lr, x_data, y_data, cv=4)
print(Rcross)
print("The mean of the folds are", Rcross.mean(), "and the standard deviation is" , Rcross.std())
Output
Mean square error using test data is 47.91753944668556
R-Square value using test data is 0.5784951912105682
Mean square error using training data is 27.636206385020866
R-Square value using training data is 0.6375993387266081
[ 0.60217169 0.60398145 0.35873597 -1.10867706]
The mean of the folds are 0.11405301290098091 and the standard deviation is 0.7129565737131626*
