How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a term-document matrix or to create anther matrix. I don't know the exact steps to follow to create training data.


You proceed the same way you would without LSA:

  1. take some labeled corpus (e.g. 20 newsgroups)
  2. preprocess (remove stop words, stem, etc)
  3. convert to the vector space model, weight by tf-idf
  4. do LSA
  5. build a classifier

So the only additional step here is the 4th one.

Here's an example for scikit learn adapted from here:

dataset = fetch_20newsgroups(subset='all', shuffle=True, random_state=42)
labels = dataset.target
true_k = np.unique(labels).shape[0]

hasher = HashingVectorizer(n_features=opts.n_features,
                           stop_words='english', non_negative=True,
                           norm=None, binary=False)
vectorizer = make_pipeline(hasher, TfidfTransformer())
X = vectorizer.fit_transform(dataset.data)

svd = TruncatedSVD(true_k)
lsa = make_pipeline(svd, Normalizer(copy=False))

X = lsa.fit_transform(X)

clf = MultinomialNB().fit(X, labels)

Here's a list of useful links on how to work with textual data in scikit learn:

  • $\begingroup$ I am not able to understand why you passed "true_k" i.e. labels to TruncatedSVD. It is supposed to take number of components and algorithm as arguments. $\endgroup$ – Pale Blue Dot May 23 '17 at 9:41
  • $\begingroup$ No particular reason $\endgroup$ – Alexey Grigorev May 24 '17 at 10:05
  • $\begingroup$ @AlexeyGrigorev are you sure this works? I don't see how MultinomialNB would work on negative values, and it looks like the output of the lsa step could certainly have negative values after the l2-norm is applied. $\endgroup$ – alichaudry Nov 14 '18 at 16:52
  • $\begingroup$ I copied that from the documentation. Now when you say it, I am not sure. Any other classifier would do, I suppose $\endgroup$ – Alexey Grigorev Nov 15 '18 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.