# For MSE equation does order of $y$ and $\hat{y}$ in the residual $(y-\hat{y})$ matter?

So the equation for MSE is $$\frac{1}{2N}\sum(y-\hat{y})^2$$. If you switch the order as in $$\frac{1}{2N}\sum(\hat{y} - y)^2$$ does that affect anything? The only thing I think it potentially effects is when you're doing gradient descent you have to change the sign in front of the learning rate multiplied by the derivative.

## 1 Answer

No because $$a^2 = [(-1)(-a)]^2 = [(-1)^2(-a)^2] = (-a)^2$$.

For the gradient, you'd have $$2(-a)\left[\frac{\partial}{\partial \theta}(-a) \right] = 2a \left[\frac{\partial}{\partial \theta} a \right]$$ due to the chain rule.

• Can you explain how the bears on the gradient of $(y-\hat{y})^2$?
– Sycorax
Commented Jul 22, 2021 at 4:22
• @Sycorax I just added an explanation for the gradient. The minus sign is corrected by the application of the chain rule.
– Cat
Commented Jul 22, 2021 at 4:24
• this wouldn't be the case if $\hat{y}$ is a function and you're taking the derivative for the variable correct? Like $\hat{y}=ax+b$ and you're getting the derivative w.r.t to a. Commented Jul 22, 2021 at 5:26