Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My issue is a bit hard to explain in this question's title, so hopefully I can make clear what my problem is about in this text.

I'm dealing with partially supervised text classification. I have a set of positive documents and a set of unlabeled documents (which contains both positive and negative documents). My goal is to identify documents in this unlabeled set which are most probably negative documents. As soon as I've identified them, I use those and the positive set to classify the rest of the unlabeled documents.

In order to identify the set of reliable negative documents, I use a special version of the Rocchio classification algorithm which is explained in this paper:

Xiao-Li Li, Bing Liu, See-Kiong Ng (2010) Negative Training Data can be Harmful to Text Classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-2010).

In the upper left corner of page 6 (figure 4), there is pseudocode explaining the algorithm. Below, are the two relevant lines that my question is about. $$ \begin{aligned} 3.~~~~~p &= \alpha\frac{1}{|P|}\sum_{d_j\in P}\frac{d_j}{||d_j||}-\beta\frac{1}{|PN|}\sum_{d_i\in PN}\frac{d_i}{||d_i||} ; \\ 4.~~~~~n &= \alpha\frac{1}{|PN|}\sum_{d_i\in PN}\frac{d_i}{||d_i||}-\beta\frac{1}{|P|}\sum_{d_j\in P}\frac{d_j}{||d_j||} ; \end{aligned} $$ At this stage of the algorithm, I have a set P of positive documents and a set PN of potential negative documents which were identified in a previous step. Each document in these two sets is represented as a vector (in bold letters) of TF-IDF values of the word vocabulary from the respective set. In the first line of the code above, I subtract the PN-vector from the P-vector. In the second line, I do it vice versa. The goal is to create a positive prototype vector p and and a negative prototype vector n.

My question is the following:

Which vocabulary do I have to take into account for each of these two subtractions? Do I have to create all feature vectors from the entire vocabulary of both the positive and the potential negative set? Or do I have to use only the vocabulary from the positive set in the first line and the vocabulary from the potential negative set in the second line? Or anything completely different from that? Unfortunately, this isn't explained anywhere.

share|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.