1
$\begingroup$

The base least squares formula that makes sense to me is: $$\sum(y^{(i)}-h_{\theta}(x^{(i)}))^2$$ Where $h_{\theta}$ is the hypothesis function, $y^{(i)}\in R, x^{(i)} \in R$. Then sometimes I see the function multiplied by $\frac{1}{m}$ where $m$ is the number of data points, which makes sense that's taking an average, but other times I see it just multipled by $\frac{1}{2}$ like in these Stanford ML notes: http://ufldl.stanford.edu/tutorial/supervised/LinearRegression/. But I don't really know the purpose of this, besides it cancels out the 2 when you take the derivative of the loss, but that doesn't really seem to provide much utility and I can't really think of any other benefit it gives.

$\endgroup$
1
  • 1
    $\begingroup$ It's literally just to cancel out the 2 when you take the derivative. Makes the maths a little bit more straightforward because you don't have a factor of 2 floating around. Also makes it look a bit more like the log-likelihood of a Gaussian which is useful for if you assume data form Gaussian iid errors around the hypothesis function $\endgroup$
    – jcken
    Jan 24 at 9:49
1
$\begingroup$

it cancels out the 2 when you take the derivative of the loss

That's it. This is the reason.

$\endgroup$
0
$\begingroup$

Just for mathematical convenience. Later on, you will be taking the derivative of this mean squares, so by multiplying by $\frac{1}{2}$ beforehand, you are getting rid of the factor of 2.

$\endgroup$
1
  • $\begingroup$ I feel like it's a pretty simple equation that doesn't really need to be made more convenient unless your hypothesis function is something insane. $\endgroup$ Jan 24 at 10:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.