# Using LSA for dimension reduction on test data

I am looking to explore using LSA as a dimension reduction technique for some textual data. To be specific I would like to take a matrix of M documents and N n-grams (ie: variables) and then create a smaller set of D variables that accounts for the most of the variance (thus resulting in a MxD matrix). The problem I am having is that as far as I can see lsa's fold_in() function only seems to return a full MxN matrix (albeit one that was estimated using D number of singular values).

Here is an example of what I mean:

> library(lsa)
>
> #make training set
> td = tempfile()
> dir.create(td)
> write( c("dog", "cat", "mouse"), file=paste(td, "D1", sep="/") )
> write( c("ham", "mouse", "sushi"), file=paste(td, "D2", sep="/") )
> write( c("dog", "pet", "pet"), file=paste(td, "D3", sep="/") )
> write( c("dog", "pet", "bird","cat"), file=paste(td, "D4", sep="/") )
> train = textmatrix(td)
>
> #make test set
> td = tempfile()
> dir.create(td)
> write( c("dog", "mouse","bird"), file=paste(td, "D1", sep="/") )
> write( c("ham", "mouse", "sushi"), file=paste(td, "D2", sep="/") )
> write( c("dog", "pet", "pet","cat"), file=paste(td, "D3", sep="/") )
> test = textmatrix(td)
>
> #run LSA withing training data, reducing dimension to 2
> train.lsa = lsa(train, dims=2)
>
> #apply LSA to test data
> (test.lsa <- t(fold_in(test, train.lsa)))

docs       cat       dog      mouse         ham       sushi        pet       bird
D1 0.6477117 0.7120160 0.85915463  0.43561856  0.43561856  0.3527842 0.22417560
D2 0.4613586 0.2370914 1.03194173  0.59632318  0.59632318 -0.4227943 0.02574004
D3 0.5074166 0.9633718 0.04236298 -0.08829291 -0.08829291  1.2886710 0.37676074


As you can see the resulting test.lda object is 3x7 whereas I am looking to reduce the number of number of variables to 2. I could apply prcomp() to as.textmatrix(train.lsa) and then predict.prcomp() to test.lsa to reduce dimension after the fact, but that seems redundant given that SVD is already being used to run the LSA and that only 2 singular values are being used to create test.lsa to being with. I assume that I could also, after reviewing a bit of linear algebra, use the SVD components stored within the train.lsa object to do some manual matrix multiplication to get what I'm looking for but the lack of an easy to use function in the lsa package to do this makes me weary that I'm missing something here. So my question is what am I missing? Is there a simple function that I just can't find or am I misunderstanding how LSA is supposed to be used for "dimension reduction"?

Also I apologize if this question is better suited for cross-validated - I'm kind of hoping this just a technical issue of me not seeing some function or parameter in the lsa package so I'm going with stackoverflow to start with.

-

## migrated from stackoverflow.comJul 9 '13 at 2:16

This question came from our site for professional and enthusiast programmers.

What's LSA? Presumably all of us are open to learning something new, so it's best not to assume that everyone knows your abbreviations. –  Nick Cox Jul 10 '13 at 14:10
Latent Semantic Analysis –  David Jul 10 '13 at 15:17

No bites so I'd thought I'd post + accept what I've come up until someone, hopefully,can do better.

I think that this new fold_in() function will do what I want:

fold_in2 <- function(train.lsa,test){
return(crossprod(t(crossprod(test, train.lsa$tk)), solve(diag(train.lsa$sk))))
}

> fold_in2(train.lsa,test)

docs        [,1]       [,2]
D1 -0.37404506  0.5325658
D2 -0.06960317  0.7690764
D3 -0.58860967 -0.1680743


My only hesitation with it is that I'm not 100% on the linear algebra behind it so I will likely just use straight up PCA via prcomp() - after looking at the actual code used for lsa() I'm not seeing anything special that makes it better than PCA to begin with.

-