I have to train a classification model with 15 classes based on data which contains both textual and numeric data. For instance: product description(textual), product length'(numeric). I have experience with Text mining but only with textual data. My approach would be to separate the textual and numeric data, create dfm and then merge it with numeric data. But I am open to other better approaches.

  • $\begingroup$ You can also convert numeric data to text data or a better option would be to convert text data to factor variables so that it is represented numerically $\endgroup$ – Ayush Nigam Apr 10 '18 at 10:14
  • $\begingroup$ Is it a possibility to convert textual data to numeric by calculating product descriptions properties like length of description, number of words etc, what actually represent your concerns about the text? $\endgroup$ – Asqan Apr 10 '18 at 11:34
  • $\begingroup$ the target variable is highly dependent on the content of product description. I don't think so calculating number of words would help. If I create a Document term matrix then there will be a lot of independent variables. $\endgroup$ – Aakash Apr 10 '18 at 11:51
  • $\begingroup$ What exactly is the problem in here..? No matter what algorithm you use, you need to provide the textual data in some kind of numeric representations (e.g. one-hot encoding etc.), so there is no problem with mixing both kinds of data. The only problem, for some algorithms, could be different scales, but this can be solved by scaling or normalization. $\endgroup$ – Tim Apr 11 '18 at 20:18

You have two main options here:

  1. As you said, create some numeric features out of the text description and merge it with the rest of the numeric data. The features created out of the text description can be either the document-term matrix (with tf-idf or not), can be SVD components or even averaged word-vectors (look for word2vec etc).

  2. You can build two separate classifiers (one using text data only and one using numeric only) and then combine their output using some meta-modelling.

  • $\begingroup$ Thanks. The second option looks promising but I have to do some research as I have never done meta modelling. $\endgroup$ – Aakash Apr 10 '18 at 13:47
  • $\begingroup$ It's not anything too difficult. You just have to take care to not overfit the training data. You can to properly construct CV predictions for each train fold and then build a 2nd level model using the 1st level models predictions as input features. $\endgroup$ – Stergios Apr 10 '18 at 13:52
  • 1
    $\begingroup$ You could also build 3 models. One on text data, one on numeric data (similar to point 2 above) and one on their combination (similar to point 1 above) and then combine these 3 models using a meta-modeling approach. $\endgroup$ – Stergios Apr 10 '18 at 14:34
  • 2
    $\begingroup$ I built 2 models using your 2nd approach and merged them using their probabilities. Performed surprisingly well. Thanks.. $\endgroup$ – Aakash Jun 28 '18 at 14:20
  • 1
    $\begingroup$ @lU5er: Google for model ensembling. A great guide is this one: mlwave.com/kaggle-ensembling-guide $\endgroup$ – Stergios Aug 21 '18 at 13:45

I think there is a more satisfying solution than what has been suggested already, one that creates a single model that properly deals with the two kinds of input data and their relationship to the output class. Use a sequence model like an RNN to convert text into a kind of embedding. That embedding output is used directly as input to a dense layer that also takes the non-text data as input.

The benefit of putting this into one model is you can merely rely on backpropagation to learn the right level of dependency of the output class on the two kinds of inputs, as well as let it train the RNN jointly with the final classifier. No need to add the complexity of an ensemble.

For details, here is a good tutorial:



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.