0
$\begingroup$

There is a dataset from an old kaggle competition, https://www.kaggle.com/c/lshtc/discussion/7980 and I wanted to work on it as I am learning NLP. I have done a text classification project, this is the first time I am trying hierarchical text classification.

The format of the data is as follows:

label is an integer and corresponds to the category to which the document vector belongs. Each document vector may belong to more than one category. The pair feat:value corresponds to a non-zero feature with index feat and value value. feat is an integer representing a term and value is a double that corresponds to the weight (tf) of the term in the document.

For example:

545, 32 8:1 18:2 corresponds to a document vector whose features are all zeros except feature number 8 (with value 1) and feature number 18 (with value 2). This document vector belongs to categories 545 and 32. Each feature number is associated to a stemmed word.

Can someone please explain how to use the data as input to a machine learning model? I am unsure how to take the values of the features into account. Also the documents are of different lengths. Am I supposed to create a matrix of all features and fill in the respective value counts for each document and use this matrix as input to a machine learning model?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.