# Latent semantic classification

How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a term-document matrix or to create anther matrix. I don't know the exact steps to follow to create training data.

You proceed the same way you would without LSA:

1. take some labeled corpus (e.g. 20 newsgroups)
2. preprocess (remove stop words, stem, etc)
3. convert to the vector space model, weight by tf-idf
4. do LSA
5. build a classifier

So the only additional step here is the 4th one.

Here's an example for scikit learn adapted from here:

dataset = fetch_20newsgroups(subset='all', shuffle=True, random_state=42)
labels = dataset.target
true_k = np.unique(labels).shape[0]

hasher = HashingVectorizer(n_features=opts.n_features,
stop_words='english', non_negative=True,
norm=None, binary=False)
vectorizer = make_pipeline(hasher, TfidfTransformer())
X = vectorizer.fit_transform(dataset.data)

svd = TruncatedSVD(true_k)
lsa = make_pipeline(svd, Normalizer(copy=False))

X = lsa.fit_transform(X)

clf = MultinomialNB().fit(X, labels)


Here's a list of useful links on how to work with textual data in scikit learn:

• I am not able to understand why you passed "true_k" i.e. labels to TruncatedSVD. It is supposed to take number of components and algorithm as arguments. – Pale Blue Dot May 23 '17 at 9:41
• No particular reason – Alexey Grigorev May 24 '17 at 10:05
• @AlexeyGrigorev are you sure this works? I don't see how MultinomialNB would work on negative values, and it looks like the output of the lsa step could certainly have negative values after the l2-norm is applied. – alichaudry Nov 14 '18 at 16:52
• I copied that from the documentation. Now when you say it, I am not sure. Any other classifier would do, I suppose – Alexey Grigorev Nov 15 '18 at 11:05