# LDA projection for classification

I am dealing with 2 class LDA classification problem.

During a test phase (after training), I'm trying to project a feature vector to lower dimensional space.

How do we get the projected test feature vector?

Is it

1. Y = (X-mean) * W
2. Y = X * W

Which one of above is true? (X is feature vector, W is weight vector obtained during training, Y is the resultant projected vector).

-
The following link says we just multiply feature vector with Weight vector. However, I have seen others actually subtracting mean from the feature vector, then multiplying with weight vector. en.m.wikipedia.org/wiki/Linear_Discriminant_Analysis#section_1 –  raul_w Apr 11 '12 at 23:31
For two-class LDA, wouldn't Y be just a scalar (or, if you will, a one-dimensional vector)? My best guess is, you may be confusing a dimensionality reduction PCA step (sometimes done before LDA) with LDA itself. –  SheldonCooper Apr 12 '12 at 3:28
Yes, resultant Y will be a 1-d vector. So, subtracting mean isn't necessary? –  raul_w Apr 12 '12 at 11:20
add comment

## 1 Answer

(You probably found out by now, but in case someone else needs this:)

Centering the data is independent of the projection (LDA projects into a $n_\text{classes} - 1$ dimensional space, and it doesn't matter at all wheter this is one or more dimensions).

Generally speaking, translation (i.e. using a different center) doesn't change the predictions of an LDA model as they depend on the distance between the classes in LD space. This means that implementations of LDA are free to choose whatever centering they prefer. So if and exactly what center you need to subtract will depend on the implementation of LDA you use.

As an example, MASS::lda in R uses the mean of the class means weighted by the prior probabilities:

means <- colSums(prior * object$means) scaling <- object$scaling
x <- scale(x, center = means, scale = FALSE) %*% scaling


To test this:

> LDA <- lda (Species ~ ., data = iris, prior=c (.8, .1, .1))
> plot (predict (LDA)$x, asp = 1) > LDscores <- scale (iris [, -5], center = colSums (LDA$prior * LDA$means), scale = FALSE) %*% LDA$scaling
> points (LDscores, pch = 20, col = 2)


> summary (LDscores - predict (LDA)\$x)
LD1         LD2
Min.   :0   Min.   :0
1st Qu.:0   1st Qu.:0
Median :0   Median :0
Mean   :0   Mean   :0
3rd Qu.:0   3rd Qu.:0
Max.   :0   Max.   :0

-
add comment