2
$\begingroup$

I am trying to learn gradient descent and in the course of so I am trying to find the optimal m and c value for my model, for $y=mx+c$

For that, I have plotted the MSE using the below code in python

   #building the model

#lets start with random value of m and c

m=0
c=0

l=0.0001 #l is the learning rate


n=float(len(X_train)) # n=number of training data, we are converting it to float since we will need to divide n

mse=[]

for i in range (0,4000):
    Y_pred=m*X_train+c
    
    mse.append(numpy.sum((Y_train-Y_pred)**2)/n)
    D_m= (-2/n) * sum(X_train*(Y_train-Y_pred))
    D_c= (-2/n) * sum(Y_train-Y_pred)
    
    m=m-l*D_m
    c=c-l*D_c
   
 
    
plt.plot(mse)

And the output I am getting for this is this-->

enter image description here

So it seems that the SME becomes more or less same after 2000, and remains more or less same till 4000

So I am taking m and c values that I have got in 4000th iteration. From the graph, we can see that the MSE value is lesser than 0.2.

But to my surprise when I do

mse[-1]

I get a HUGE NUMBER as Answer

The answer that I get for mse[-1] is 12041739532.188858

And for this reason, my final model performs absolutely worst, producing something like this as the output on the training set.

enter image description here

It will be very helpful if someone can guide me on why this is happening with the MSE value. Thank you.

$\endgroup$
1
$\begingroup$

Look at the scale of your graph. The vertical axis spans something like $10^{10}$ to $10^{11}$, as you can see by the 1e11 printed above the vertical scale. Your plot doesn't show that the error is below 0.2, it shows that the error is below 0.2e11.

This is consistent with the printed value for mse[-1].

$\endgroup$
14
  • 1
    $\begingroup$ yeah, this is one of those instances where the way the default choices of the plotting package seem almost deliberately deceptive. $\endgroup$ – Sycorax Aug 11 '20 at 17:00
  • $\begingroup$ what should i do to converge the MSE much faster? increase the learning rate right? $\endgroup$ – Turing101 Aug 11 '20 at 17:01
  • $\begingroup$ I watched this here--> youtube.com/watch?v=4PHI11lX11I $\endgroup$ – Turing101 Aug 11 '20 at 17:05
  • $\begingroup$ @carlo, can u explain where it is wrong? $\endgroup$ – Turing101 Aug 11 '20 at 17:09
  • 1
    $\begingroup$ Guys figured out at last why it was giving such weird values. I haven't feature scaled my data. That is why it was acting like this after feature scaling everything is working fine as expected. $\endgroup$ – Turing101 Aug 12 '20 at 6:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.