0
$\begingroup$

Referring to the below diagram from Chris Bishop (with my markings/write-ups in yellow color), I tried to find the distance (marked as $r$) of a point $\mathbf{x}$ from decision surface (the red line).

When I consider the point $\mathbf{x}$ from $O$ as origin instead of the origin of $x_1$ and $x_2$, I could derive (see after the diagram) what is written in the book. My concern is that if this change of origin is correct? From the blue coloring of $\mathbf{x}$, it seems the vector is supposed to be considered from the origin of the $x_1$ and $x_2$. If so, then how do we arrive at the same derivation for the distance $r$?

enter image description here

My derivation:

$$ \mathbf{x} = \mathbf{x_\perp} + r \frac{\mathbf{w}}{||\mathbf{w}||}$$ Given Linear discriminant function $ y(\mathbf{x})=\mathbf{w}^T\mathbf{x} + w_0 $, and since $\mathbf{x_\perp}$ is on the decision-boundary $y(\mathbf{x})=0$ (the red line), we have $$ y(\mathbf{x_\perp})=0 = y(\mathbf{x} - r \frac{\mathbf{w}}{||\mathbf{w}||}) = \mathbf{w}^T(\mathbf{x} - r \frac{\mathbf{w}}{||\mathbf{w}||}) + w_0 $$ Solving for $r$, $$ r=\frac{\mathbf{w}^T\mathbf{x} + w_0}{||\mathbf{w}||} = \frac{y(\mathbf{x})}{||\mathbf{w}||} $$

$\endgroup$
  • $\begingroup$ Isn't that distance r the discriminant score? i.e., the projection of deviation vector X onto discriminant direction w, that vector w being translated to start from the red boundary line, even from the centroid of the data cloud O? Discriminant analysis computes discriminant scores for every point, it is one of its tasks. $\endgroup$ – ttnphns Dec 15 '19 at 16:20
0
$\begingroup$

Someone just pointed me out that we don't need to shift the origin as I suspected. The equation: $$ \mathbf{x} = \mathbf{x_\perp} + r \frac{\mathbf{w}}{||\mathbf{w}||}$$

is still valid considering the $\mathbf{x}$ (blue-line) in original coordinate along with $\mathbf{x_\perp}$ as the vector shown in fuchsia-line (see below).

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.