# Calculations of linear discriminant function for classification

Referring to the below diagram from Chris Bishop (with my markings/write-ups in yellow color), I tried to find the distance (marked as $$r$$) of a point $$\mathbf{x}$$ from decision surface (the red line).

When I consider the point $$\mathbf{x}$$ from $$O$$ as origin instead of the origin of $$x_1$$ and $$x_2$$, I could derive (see after the diagram) what is written in the book. My concern is that if this change of origin is correct? From the blue coloring of $$\mathbf{x}$$, it seems the vector is supposed to be considered from the origin of the $$x_1$$ and $$x_2$$. If so, then how do we arrive at the same derivation for the distance $$r$$?

My derivation:

$$\mathbf{x} = \mathbf{x_\perp} + r \frac{\mathbf{w}}{||\mathbf{w}||}$$ Given Linear discriminant function $$y(\mathbf{x})=\mathbf{w}^T\mathbf{x} + w_0$$, and since $$\mathbf{x_\perp}$$ is on the decision-boundary $$y(\mathbf{x})=0$$ (the red line), we have $$y(\mathbf{x_\perp})=0 = y(\mathbf{x} - r \frac{\mathbf{w}}{||\mathbf{w}||}) = \mathbf{w}^T(\mathbf{x} - r \frac{\mathbf{w}}{||\mathbf{w}||}) + w_0$$ Solving for $$r$$, $$r=\frac{\mathbf{w}^T\mathbf{x} + w_0}{||\mathbf{w}||} = \frac{y(\mathbf{x})}{||\mathbf{w}||}$$

• Isn't that distance r the discriminant score? i.e., the projection of deviation vector X onto discriminant direction w, that vector w being translated to start from the red boundary line, even from the centroid of the data cloud O? Discriminant analysis computes discriminant scores for every point, it is one of its tasks. – ttnphns Dec 15 '19 at 16:20

## 1 Answer

Someone just pointed me out that we don't need to shift the origin as I suspected. The equation: $$\mathbf{x} = \mathbf{x_\perp} + r \frac{\mathbf{w}}{||\mathbf{w}||}$$

is still valid considering the $$\mathbf{x}$$ (blue-line) in original coordinate along with $$\mathbf{x_\perp}$$ as the vector shown in fuchsia-line (see below).