So the equation for MSE is $\frac{1}{2N}\sum(y-\hat{y})^2$. If you switch the order as in $\frac{1}{2N}\sum(\hat{y} - y)^2$ does that affect anything? The only thing I think it potentially effects is when you're doing gradient descent you have to change the sign in front of the learning rate multiplied by the derivative.


1 Answer 1


No because $a^2 = [(-1)(-a)]^2 = [(-1)^2(-a)^2] = (-a)^2$.

For the gradient, you'd have $2(-a)\left[\frac{\partial}{\partial \theta}(-a) \right] = 2a \left[\frac{\partial}{\partial \theta} a \right]$ due to the chain rule.

  • $\begingroup$ Can you explain how the bears on the gradient of $(y-\hat{y})^2$? $\endgroup$
    – Sycorax
    Jul 22, 2021 at 4:22
  • $\begingroup$ @Sycorax I just added an explanation for the gradient. The minus sign is corrected by the application of the chain rule. $\endgroup$
    – Cat
    Jul 22, 2021 at 4:24
  • $\begingroup$ this wouldn't be the case if $\hat{y}$ is a function and you're taking the derivative for the variable correct? Like $\hat{y}=ax+b$ and you're getting the derivative w.r.t to a. $\endgroup$ Jul 22, 2021 at 5:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.