There is a dataset from an old kaggle competition, https://www.kaggle.com/c/lshtc/discussion/7980 and I wanted to work on it as I am learning NLP. I have done a text classification project, this is the first time I am trying hierarchical text classification.

The format of the data is as follows:

label is an integer and corresponds to the category to which the document vector belongs. Each document vector may belong to more than one category. The pair feat:value corresponds to a non-zero feature with index feat and value value. feat is an integer representing a term and value is a double that corresponds to the weight (tf) of the term in the document.

For example:

545, 32 8:1 18:2 corresponds to a document vector whose features are all zeros except feature number 8 (with value 1) and feature number 18 (with value 2). This document vector belongs to categories 545 and 32. Each feature number is associated to a stemmed word.

Can someone please explain how to use the data as input to a machine learning model? I am unsure how to take the values of the features into account. Also the documents are of different lengths. Am I supposed to create a matrix of all features and fill in the respective value counts for each document and use this matrix as input to a machine learning model?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.