3
$\begingroup$

I've followed the Machine Learning course of Andrew Ng, and I really confuse in Support Vector Machine lecture. Regarding cost function in SVM, he said that when C is very large, the loss (error) equal to 0. Can you guys help me to explain why is that? enter image description here

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
2
$\begingroup$

If $C$ is very large, in order to reduce the cost, the term that is multiplied with $C$ should be very small, e.g. in the extreme case, if $C$ is infinite, then the other term should be $0$, otherwise the cost will be infinity.

Consider the following simple cost function:

$$J(\theta)=C\theta+f(\theta)$$

If $C$ is very very large, the optimization algorithm doesn't care about the second term much and tries to minimize the first one, e.g. $\theta$ will be close o $0$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So, does it also mean if C increase, it'll lead to cost increase and underfitting, and if C decrease, then cost'll decrease and overfitting? $\endgroup$
    – Thuan Do
    May 27, 2020 at 19:10
  • $\begingroup$ that is correct $\endgroup$
    – gunes
    May 27, 2020 at 19:11
  • $\begingroup$ thank you so much. $\endgroup$
    – Thuan Do
    May 27, 2020 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.