-1
$\begingroup$

I have a set of image documents. I extract text keywords from this images using OCR to represent each image as a bag of words (a vector where each value is the number of occurrence of a word in the document). Then I can apply a classification or clustering algorithm on the obtained dataset.

However, this vector representation as bag of words is possible only if I have the entire set of documents (to be able to have the entire vocabulary, i.e. all words). How can I do that (i.e. extract my bag of words) if I'm in an online configuration (using an online clustering) where the documents are available one by one (for a data stream) and each document should be processed as soon as it is available ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

If you use sparse vector formats, it should not be a problem to add new dimensions later on. In sparse vectors, unset dimensions default to 0, usually. Since that is "word not present" it should be fine.

So, have you tried it? Where exactly does the problem arise?

Improve this answer
$\endgroup$
8
  • $\begingroup$ Is there any other formats for bag of words than the "sparse vector formats" ? If I add a new dimension each time I find a new word, we will have an unbounded dimensionality. is it possible to dynamically keep only the significant words, as we go ? $\endgroup$
    – shn
    Commented Oct 30, 2012 at 15:43
  • $\begingroup$ Why do you need a finite dimension for a sparse vector format? $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Oct 30, 2012 at 15:49
  • $\begingroup$ Because the classical distances become weakly discriminant when we have a highly multidimensional and sparse data $\endgroup$
    – shn
    Commented Oct 30, 2012 at 15:57
  • $\begingroup$ Pretending that you have fewer dimensions does not change this at all. It does not actually change the distance values if you leave away zeros! Plus, you can keep a global list of dimensions to skip, i.e. project your data to some subspace. This does not at all collide with sparse vectors. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Oct 30, 2012 at 16:00
  • $\begingroup$ Ok, then I'll use sparse vector format. Another question would be: is it more difficult (compared with low dimensional dense data) to learn a distance metric function (say a vector of weights for a weighted euclidean distance) using some labelled data, when the data-points are actually bag of words (multidimensional and sparse vectors) ? $\endgroup$
    – shn
    Commented Oct 30, 2012 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.