I have been trying to implement a linear regression model on my own with multiple variables.

After implementing it, I see that my loss doesn't decrease.
I think that I'm facing a dimension problem.
Here is my code:

def initialize_parameters(X): 
    n = X.shape[1]
    return np.random.randn(n,1)

def compute_cost(Y,X,W): 
    m = X.shape[0]
    return (1/2*m) * np.sum(np.power(np.dot(X,W) - Y,2))

def gradient_descent(Y,X,W,epochs,learning_rate): 

    m = X.shape[0]
    cost_history = [] 
    W_temp = W.copy()
    for i in range(epochs): 

        grads = (learning_rate/m) * np.dot(X.T, np.dot(X,W) -Y)
        W_temp = W_temp - grads 
        cost = compute_cost(Y,X,W_temp) 
    return W_temp, cost_history 

W = initialize_parameters(X_norm)
W_best, cost_history = gradient_descent(Y,X_norm,W, 2000, 0.003)
plt.plot([i for i in range(2000)], cost_history)


1 Answer 1


Okay so I fixed it. When doing the gradient descent, I just replaced

w_temp = w.copy() 

by directly updating w such as :

w = w - grad

Now it works !

  • $\begingroup$ That's great! But it appears that your problem was purely a programming problem, and so it is not suitable for a statistics site. Can you delete your question? $\endgroup$ Commented Aug 17, 2020 at 6:09
  • $\begingroup$ cannot delete because of the edit saldy $\endgroup$
    – Valentin
    Commented Aug 19, 2020 at 10:50
  • $\begingroup$ Oh. I flagged a moderator to help. $\endgroup$ Commented Aug 19, 2020 at 11:19
  • 1
    $\begingroup$ There is no need for this Q to be deleted. It is closed, but it can continue to exist in that state. If you want your Q migrated to Stack Overflow, flag it for moderator assistance. $\endgroup$ Commented Aug 19, 2020 at 11:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.