I have been trying to implement multiple linear regression from scratch after implementing it using sklearn. The values predicted using sklearn is very accurate whereas the values predicted by the code which I have coded from scratch is not so much accurate. This is the dataset that I am using (I have provided the entire dataset in my Github link mentioned below along with the source code of my implementation)
I am predicting profit of an organisation by seeing the 4 independent variables. Notice that we also have a categorical value here, on which I have done 1 hot encoding, and I have also scaled all the data.
I am using gradient descent to reach the optimal values for slope for the independent variables. The function I have written for gradient descent updation is as follows.
#building the model #lets start with a random value of m and c m_1=0 m_2=0 m_3=0 m_4=0 m_5=0 m_6=0 c=0 l=0.0001 #l is the learning rate n=float(len(X_train)) # n=number of training data, we are converting it to float since we will need to divide n mse= #we are creating a list for storing mean square errors for each iteration so that we can plot it for i in range (0,20000): Y_pred=m_1*X_train[:,0:1]+m_2*X_train[:,1:2]+m_3*X_train[:,2:3]+m_4*X_train[:,3:4]+m_5*X_train[:,4:5]+m_6*X_train[:,5:6]+c mse.append(numpy.sum((Y_pred-Y_train)**2)/n) D_m_1= (2/n) * sum(X_train[:,0:1]*(Y_pred-Y_train)) D_m_2= (2/n) * sum(X_train[:,1:2]*(Y_pred-Y_train)) D_m_3= (2/n) * sum(X_train[:,2:3]*(Y_pred-Y_train)) D_m_4= (2/n) * sum(X_train[:,3:4]*(Y_pred-Y_train)) D_m_5= (2/n) * sum(X_train[:,4:5]*(Y_pred-Y_train)) D_m_6= (2/n) * sum(X_train[:,5:6]*(Y_pred-Y_train)) D_c= (2/n) * sum(Y_pred-Y_train) #Andrew Ng has not taken 2/n but instead 1/n as his cost function is multiplied by 1/2 to reduce complexity m_1=m_1-l*D_m_1 m_2=m_2-l*D_m_2 m_3=m_3-l*D_m_3 m_4=m_4-l*D_m_4 m_5=m_5-l*D_m_5 m_6=m_6-l*D_m_6 c=c-l*D_c plt.plot(mse) #see the plot below, we can see the point where MSE stabilises. Run the above loop till that point and take MSE
This is the function that I have written, and in the other code where I have used sklearn library the code looks like this,
#now we will fit the linear regression model from sklearn.linear_model import LinearRegression import matplotlib.pyplot as plt regressor= LinearRegression() regressor.fit(X_train,Y_train) #predicting the test results Y_pred=regressor.predict(X_test)
But there is a significant difference between the predicted value of the 2 code and as obvious the sklearn is producing good results. I am not getting why the code which I have written from scratch is having low accuracy. I am providing the dataset as well as the code (where I have written the gradient descent) below.
Link to code and dataset--> https://github.com/Hirak999/StackExchange-Doubts