5
$\begingroup$

I am using LDA (Latent Dirichlet Allocation) to extract topics. I want to do topic modelling and use the topics as features to do document classification. the reason for doing classification is to evaluate my LDA model.

the same as this link lda , my question is that when I generated my topics, what should be the labels for the classification method?

I was supposed it has to be the topics generated by the LDA, would you please provide me with your idea, suppose we have six topics generated:

topi0: computer, internet, net, technology topic1: politicians, political, barack, topic2: music, hobby,... topic3:... topic4: topic5: ... topic6: ...

so what is the labels for the classification part?

sv=SVC()
labels = [???]

sv.fit(lda_x,labels)
predictclass = sv.predict(lda_x)

testLables=[???]
from sklearn import metrics
yacc=metrics.accuracy_score(testLables,predictclass)

Update

my dataset is 20news group, my aim is to use LDA to extract for example 6 topic(also each topic has its own distribution of terms) then use one classification method to evaluate that.

Update 2 code part this is the output for the matric to send to classification method. I now think there is something wrong with that,

[[ 9.29756948e-07 9.99991632e-01 9.29752853e-07 9.29718960e-07 9.29696185e-07 9.29741802e-07 9.29735986e-07 9.29749709e-07 9.29697295e-07 9.29716150e-07]]

from __future__ import print_function

import nltk
import numpy as np

from time import time

import gensim
from gensim import corpora
from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer
from sklearn.decomposition import NMF, LatentDirichletAllocation
from sklearn.datasets import fetch_20newsgroups
from nltk.tokenize import RegexpTokenizer, sent_tokenize, word_tokenize
from nltk.stem.porter import PorterStemmer

from sklearn.naive_bayes import MultinomialNB
from sklearn.preprocessing import LabelEncoder

n_samples = 2000
n_features = 1000
n_topics = 10
n_top_words = 20


def print_top_words(model, feature_names, n_top_words):
    for topic_idx, topic in enumerate(model.components_):
        print("Topic #%d:" % topic_idx)
        print(" ".join([feature_names[i]
                        for i in topic.argsort()[:-n_top_words - 1:-1]]))
    print()


# Load the 20 newsgroups dataset and vectorize it. We use a few heuristics
# to filter out useless terms early on: the posts are stripped of headers,
# footers and quoted replies, and common English words, words occurring in
# only one document or in at least 95% of the documents are removed.

texts=[]
print("Loading dataset...")
t0 = time()
dataset = fetch_20newsgroups(shuffle=True, random_state=1,
                             remove=('headers', 'footers', 'quotes'))
data_samples = dataset.data[:n_samples]
print("done in %0.3fs." % (time() - t0))

# Use tf-idf features for NMF.
print("Extracting tf-idf features for NMF...")
tfidf_vectorizer = TfidfVectorizer(max_df=0.95, min_df=2,
                                   max_features=n_features,
                                   stop_words='english')
t0 = time()

tfidf = tfidf_vectorizer.fit_transform(data_samples)
p_stemmer = PorterStemmer()

str=''.join(data_samples)
sent = sent_tokenize(str)
str2=''.join(sent)
words = word_tokenize(str2)
filtered_sentence = [w for w in words]
stemmed_tokens = [p_stemmer.stem(i) for i in filtered_sentence]
# add tokens to list
texts.append(stemmed_tokens)

values = ' '.join(' '.join(elems) for elems in texts).lower()
print (values)

print("done in %0.3fs." % (time() - t0))

# Use tf (raw term count) features for LDA.
print("Extracting tf features for LDA...")
tf_vectorizer = CountVectorizer(max_df=3, min_df=0.02,
                                max_features=n_features,
                                stop_words='english')
t0 = time()
y=[values]
tf = tf_vectorizer.fit_transform(y)
print("done in %0.3fs." % (time() - t0))

# Fit the NMF model
print("Fitting the NMF model with tf-idf features, "
      "n_samples=%d and n_features=%d..."
      % (n_samples, n_features))
t0 = time()
nmf = NMF(n_components=n_topics, random_state=1,
          alpha=.1, l1_ratio=.5).fit(tfidf)
print("done in %0.3fs." % (time() - t0))

print("\nTopics in NMF model:")
tfidf_feature_names = tfidf_vectorizer.get_feature_names()
print_top_words(nmf, tfidf_feature_names, n_top_words)

print("Fitting LDA models with tf features, "
      "n_samples=%d and n_features=%d..."
      % (n_samples, n_features))
lda = LatentDirichletAllocation(n_topics=n_topics, max_iter=5,
                                learning_method='online',
                                learning_offset=50.,
                                random_state=0)
t0 = time()
lda_x=lda.fit_transform(tf)
from sklearn.svm import SVC
for i in lda_x:
    print (i)
print("done in %0.3fs." % (time() - t0))
print("\nTopics in LDA model:")
sv=MultinomialNB()
labels = [0]

sv.fit(lda_x,labels)
predictclass = sv.predict(lda_x)

testLables=[0]
from sklearn import metrics
yacc=metrics.accuracy_score(testLables,predictclass)
print (yacc)
n=dataset.target[:20]
print (n)
$\endgroup$
  • $\begingroup$ I think you doesn't clear your classification task. The labels for classification part should come from your problems like: Sentiment analysis, Clarify/NotClarify text, ... $\endgroup$ – strnam Aug 7 '17 at 8:05
  • $\begingroup$ thankss for your answer @strnam. Actually the reason I explained about the approach is that to emphasize I am using LDA before, I was thinking any kind of dataset I have does not matter as there is 6 topic so I have to have 6 lables also. but I am not sure if this is correct or not. any way I updated. please let me know your view $\endgroup$ – sariii Aug 7 '17 at 14:02
  • $\begingroup$ if you need the whole code please let me know, i will update again $\endgroup$ – sariii Aug 7 '17 at 14:28
  • $\begingroup$ Let say the classification problem now is predict the category of news. As normal, after you trained LDA model ( 6 topics as you said) then you can apply LDA to transfer every document to 6 features ( topic distribution of document) that mean you can transfer your list document to features space with 6 feature. Finally you apply one classifier (linear regeression, svm,...) to predict categories. Hope this explain is useful. $\endgroup$ – strnam Aug 7 '17 at 19:55
  • 1
    $\begingroup$ @strnam many thanks for your complete explanations, i ve done the thing you have said except the finall part for evaluation, i mean the last line . Im not able to provide the correct lables for my classifier, it seems i lack some concept. May i ask you to provide me with the lables, im going to update withmy code, again thanks for the time you put 😊 $\endgroup$ – sariii Aug 7 '17 at 21:32
1
$\begingroup$

I assume that you refer to Section 7.2 in the original LDA paper.

In that case, as the paper suggests in the first paragraph of Section 7.2, you must have a discriminative framework; namely, a dataset with labeled documents. This is the only way to train an SVM classifier.

From you question, I infer that you use LDA in order to extract topics for a corpus with unknown topics and that you don't have any labeled data. Therefore, you simply cannot build an SVM classifier with what you have.

$\endgroup$
1
$\begingroup$

LDA topic assignments are informative as an exploratory unsupervised exercise but they are way too imperfect to use directly as source of labels for SVM. The problem is that SVM expects one or more distinct labels that align with your conceptual understanding of the subject matter while LDA provides a statistical but thoughtless mixture of topic labels based on word co-occurrences and distributions. The most you can do is use the LDA labels as a starting point to assist you in manually creating a training dataset for SVM.

$\endgroup$
0
$\begingroup$

Still not clear your question. But assume that we work on classification problem of 20newsgroups dataset, we can get:

data_samples = dataset.data[:n_samples]
labels = dataset.target[:n_samples]

data_samples contain list of document, labels contain list of category that each document in data_samples belong.

As your code, we get document-topic_distribution matrix lda_x by code:

tf_vectorizer = CountVectorizer(max_features=n_features,
                                stop_words='english')
tf = tf_vectorizer.fit_transform(data_samples)

lda = LatentDirichletAllocation(n_topics=n_topics, max_iter=5,
                                learning_method='online',
                                learning_offset=50.,
                                random_state=0)

lda_x=lda.fit_transform(tf) 

Example: lda_x[0] is a topic distribution of data_samples[0] (the probabilities document data_samples[0] belong to topics)

Now we use topic-distribution as the features to predict the category of document. It's seem that we apply any classifier to the lda_x matrix:

sv=MultinomialNB()
sv.fit(lda_x,labels)
predictclass = sv.predict(lda_x)
yacc=metrics.accuracy_score(labels,predictclass)
print (yacc)
$\endgroup$
  • $\begingroup$ many thanks for your answer, but it throws error as SV does not fit, so I added sv.fit(lda_x,labels) then it raises this error ValueError: Found input variables with inconsistent numbers of samples: [1, 2000]. may please let me know your view $\endgroup$ – sariii Aug 10 '17 at 17:52
  • $\begingroup$ I forgot the training step: sv.fit(lda_x,labels). Updated $\endgroup$ – strnam Aug 10 '17 at 18:26
  • 1
    $\begingroup$ So for achieving the label you said labels = dataset.target[:n_samples] , I think you suppose using the label have already available in newgroup, but I want to use the label in LDA being generated. I want to use the first approach being discussed here stats.stackexchange.com/questions/138352/… so I think the topic generated is my labels . an I correct? $\endgroup$ – sariii Aug 11 '17 at 1:07
  • $\begingroup$ The first approach of the link you shared is what we did. The result of LDA is matrix lda_x ("Represent each document as a vector of topic proportions. This is the feature value vector for a training document."). If you " want to use the label in LDA being generated", why you want a classifier ? It will be clarify if you define what is the finally result do you want at the end ? $\endgroup$ – strnam Aug 11 '17 at 4:11
  • $\begingroup$ I hope you can answer it soon as I really need your explanations. My ultimate goal is to evaluate my topic model. here I used LDA so I want to see how accurate the result of the topics being generated is. in my idea I have nothing to do with the labels that documents already had. its about my topic. so for testing part. tested document will be given to LDA model ... please let me know with these explanations if anything is missed? many thanks :) $\endgroup$ – sariii Aug 11 '17 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.