So the equation for MSE is $\frac{1}{2N}\sum(y-\hat{y})^2$. If you switch the order as in $\frac{1}{2N}\sum(\hat{y} - y)^2$ does that affect anything? The only thing I think it potentially effects is when you're doing gradient descent you have to change the sign in front of the learning rate multiplied by the derivative.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
No because $a^2 = [(-1)(-a)]^2 = [(-1)^2(-a)^2] = (-a)^2$.
For the gradient, you'd have $2(-a)\left[\frac{\partial}{\partial \theta}(-a) \right] = 2a \left[\frac{\partial}{\partial \theta} a \right]$ due to the chain rule.
-
$\begingroup$ Can you explain how the bears on the gradient of $(y-\hat{y})^2$? $\endgroup$– Sycorax ♦Jul 22, 2021 at 4:22
-
$\begingroup$ @Sycorax I just added an explanation for the gradient. The minus sign is corrected by the application of the chain rule. $\endgroup$– CatJul 22, 2021 at 4:24
-
$\begingroup$ this wouldn't be the case if $\hat{y}$ is a function and you're taking the derivative for the variable correct? Like $\hat{y}=ax+b$ and you're getting the derivative w.r.t to a. $\endgroup$ Jul 22, 2021 at 5:26