I am working on a problem of finding similar documents. I am using a Tf-Idf based vector space model representation of documents and it gives me good results.
However when I encounter a document with new words which is out of the current vector space, it gives an error. For example I have two documents represented as vectors of terms with their weights Document A -> (A,B,C)--> ( 0.15,0.4,0.3) Document B->(C,D,E)--> (0.12,0.3,0.5)
How to compute similarity in these situations and so on
Is there an implementation in
nltk which takes care of incrementally extending the vector space as new documents are entered.