1
$\begingroup$

Background: I have a 2-dimensional 2-class classification problem, the following training sample vectors are given:

$$D_1 = \left\{ \begin{bmatrix} 0 \\ -1 \end{bmatrix}, \begin{bmatrix} 3 \\ -2\end{bmatrix}, \begin{bmatrix} 0 \\ 2 \end{bmatrix}, \begin{bmatrix} -2 \\ 1\end{bmatrix}, \begin{bmatrix} 2 \\ -1 \end{bmatrix} \right\} $$ and $$ D_2=\left\{ \begin{bmatrix} 6 \\ 0 \end{bmatrix}, \begin{bmatrix} 3 \\ 2\end{bmatrix}, \begin{bmatrix} 9 \\ 1 \end{bmatrix}, \begin{bmatrix} 7 \\ 4\end{bmatrix}, \begin{bmatrix} 5 \\ 5 \end{bmatrix} \right\}$$

I used Fisher linear discriminant analysis method to calculate a transform vector $w$.

I wrote the code and I got

w =

    -0.2308
    -0.2033

Question: How can I calculate the projections of all the data samples in the resulting 1-dimensional space?

Do I just multiply the data with the $w$ and get new $D_1$ and $D_2$?

I found a slide, can someone explain this to me?

The projection from a $d$-D to $(c\!-\!1)$-D can be expressed as
$y_i = w_i^t x \quad i=1, \dots, c-1 \quad \text{or}$
$y=W^t x$, where $w_i$ are columns of $d \times (c\!-\!1)$ matrix $W$.

$\endgroup$
  • $\begingroup$ Did you attempt to search for answers already published? $\endgroup$ – ttnphns Mar 27 '17 at 18:00
  • $\begingroup$ I did. I am just not sure about what it means to project to 1D space? will we have a single number for each point instead of [x1 x2] $\endgroup$ – Pete Mar 27 '17 at 18:02
  • $\begingroup$ I actually found something and I added it to the question, can you explain this? $\endgroup$ – Pete Mar 27 '17 at 18:10
2
$\begingroup$

"Projection" here can be an ambiguous term. It could mean projection onto a 1D space, or onto the subspace spanned by the line. The dot product, as you mention gives the projection in the former case. In the later case, for each data point, you are essentially finding the closest point on the line to it in the original 2D space. See here for calculation of the closest point.

Here is some python code to calculate both cases.

import matplotlib.pyplot as plt
import numpy as np

D = np.matrix("0 -1; 3 -2; 0 2; -1 1; 2 -1;" +
              "6 0; 3 2; 9 1; 7 4; 5 5")

w = np.array([-0.2308, -0.2033])
slope = w[1]/w[0]

# This gives the projection onto the 1D space
np.dot(D, w)

# Plot point
plt.plot(D[0:5, 0], D[0:5, 1], 'bo')
plt.plot(D[5:, 0], D[5:, 1], 'ro')
plt.plot([-1, 10], [-slope, 10*slope], color='k', linestyle='-', linewidth=1)

# Subspace Projection https://en.wikibooks.org/wiki/Linear_Algebra/Orthogonal_Projection_Onto_a_Line
DP = dot(D, w).T * w / (dot(w,w))

# Plot projected points
plt.plot(DP[0:5, 0], DP[0:5, 1], 'bs')
plt.plot(DP[5:, 0], DP[5:, 1], 'rs')
plt.gca().set_aspect('equal', adjustable='box')
plt.draw()

enter image description here

$\endgroup$
  • $\begingroup$ Why is the slope = w[1] / w[0]? $\endgroup$ – Thomas Dec 29 '19 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.