I am working on text classification using TF/IDF and cosine similarity metrics. In computing the similarity between two records (objects) in a database, TF/IDF assigns weight vector for each record by assigning weight score to each attributes (strings) in a record. Cosine similarity is then used to measure the similarity of the two records from their weight vectors generated by TF/IDF.
I realised that TF/IDF does not give accurate weight score when typographical (spelling) errors are present in strings, this is because TF/IDF assigns accurate scores to words present in the dictionary (derived from the training set), hence wrongly spelt words are poorly scored, and results in the computation of inaccurate similarity between records.
I also realised that character-based similiarity metrics (affine gap, Jaro-winkler, levenshtien distance etc) are effective in computing the similarity between strings with typographical (spelling) errors, but are not effective when words are out of order in strings.
Hence, in order to measure the similarity between documents (records), I would like to enhance the effectiveness of TF/IDF to deal with typographical errors by using the error correcting ability of a character based similarity metric such as Levenshtein distance or affine gap or even Jaro distance.
Any suggestions will be highly appreciated.