explain the cost function of scalar vector machine (SVM) [duplicate]

I am unable to understand the above cost function. The two possible outputs ($$y$$) are $$-1$$ and $$+1$$. As far as i know, $$x_i$$ values are individual training example values, $$y_i$$ are actual true values for that example and w is the weights that we are adjusting so as to minimise the cost. if the actual output is $$1$$ and and my SVM predicts it as $$-1$$ my classifier is penalised, how will my SVM penalise? how much will be the penalty? I know this function gives cost 0 for that training example when prediction is correct and some penalty if predicted wrong. I am also confused how $$\max(0,1-y(wx+b))$$ outputs the cost for one training example. please enlighten me how this cost function works.

The following diagram I made illustrates the geometric meaning of $$w \cdot x_i + b$$ in a 2D plane: where $$w$$ is a norm vector that controls the direction of the linear boundary $$L$$; for simplicity, let's assume $$\|w\| = 1$$, then $$b$$ represents an offset applied to $$L$$ along the opposite direction of $$w$$. The distance of a point $$x_i$$ to $$L$$ is therefore $$w \cdot x_i - \left(-b\right) = w \cdot x_i + b$$.

Let $$\hat{y_i} = w \cdot x_i + b$$ be the model's prediction for sample $$i$$; then we have:

• If $$y_i = +1$$ and $$\hat{y_i} \lt +1$$, then $$\mathrm{max}\left(0, 1 - \hat{y_i}\right) = 1 - \hat{y_i} \gt 0$$;
• If $$y_i = -1$$ and $$\hat{y_i} \gt -1$$, then $$\mathrm{max}\left(0, 1 - \hat{y_i}\right) = 1 + \hat{y_i} \gt 0$$;
• If $$y_i = +1$$ and $$\hat{y_i} \ge +1$$, then $$\mathrm{max}\left(0, 1 - \hat{y_i}\right) = 0$$;
• If $$y_i = -1$$ and $$\hat{y_i} \le -1$$, then $$\mathrm{max}\left(0, 1 - \hat{y_i}\right) = 0$$;

The intuition here is that the penalty is $$0$$ only when the model's prediction is sufficiently correct (i.e. $$y_i$$ and $$\hat{y_i}$$ are of the same symbol and $$\left|\hat{y_i}\right| \gt 1$$).

• ok thank u ..i get it now...but still i have a confusion about the 1−y(wx+b) part. if i am not wrong yi in above equation is the actual output of training example xi. so if suppose yi is -1 , the equation becomes 1+(wx+b) (replacing yi =-1). how this equation will give a value less than or equal to zero so that no penalty is their for right prediction...plz give some intuition on "1−y(wx+b)" this part of equation – rushikesh chaskar Sep 12 at 4:17
• @rushikeshchaskar In the case where $y_i$ is $-1$, as you said, the second argument of the $\mathrm{max}$ function becomes $1 + \left(w x_i + b\right)$. For the loss to stay at $0$, $\left(w x_i + b\right)$ has to be less or equal to $-1$. To simplify things, let's assume we are working with 2-dimensional data, so the two parameters, $w$ and $b$ specifies a line on the 2D plane; the key here is to understand the expression $w x_i + b$ describe the "distance" from $x_i$ to the line defined by $w$ and $b$ (the unit of that distance is determined by $w$). – Xiubo Zhang Sep 12 at 11:43
• @rushikeshchaskar This "distance" has a direction. if $y_i$ is $-1$, then a "good" prediction that wouldn't incur penalty must have the same direction as $y_i$, i.e. $w x_i + b$ has to be negative; but this is not enough as SVM is a "maximum margin classifier"; this means the "distance" from $x_i$ to the linear boundary defined by the function $y = w x + b$ has to exceed the threshold of $1$ for the SVM algorithm to wavier the penalty --- when $y_i = -1$, this means $w x_i + b$ must be less than or equal to $-1$ (same direction as $y_i$ and magnitude is greater than or equal to $1$). – Xiubo Zhang Sep 12 at 11:55
• i know we shud avoid comments thanking others..but thank u so much for ur time.. i understood it well Xiubo Zhang – rushikesh chaskar Sep 12 at 16:13