Please forgive my ignorance, however I would like to explain this problem and get some advice on how to approach it.

Let's say that I have the following training inputs, where word, x, and y are the data, and class is the classification that the data has been put into, with 0 being unclassified -

Training 1

word, x, y, class
56, 48, 23, 0
91, 12, 44, 2
74, 45, 23, 0
91, 76, 48, 1

Training 2

word, x, y, class
49, 48, 45, 0
84, 16, 12, 2
10, 45, 23, 0
72, 76, 48, 3
84, 18, 12, 0
10, 45, 23, 1
24, 79, 48, 0

And I would then like to provide the following test data, to be classified -

Test 1

word, x, y
64, 36, 45
84, 16, 12
95, 45, 23
72, 76, 88
22, 18, 12

What might be the best approach to this problem?

  • 1
    $\begingroup$ What is the purpose of having two explicit training data sets? Any way to bind them together, eventually by adding a dummy variable that indicates if the line came for training 1 or 2? $\endgroup$ – Michael M Oct 31 '17 at 18:40
  • $\begingroup$ Each training data set is actually one example of a document. Each one will be similar but slightly different. They can be flattened into one file. $\endgroup$ – gamesmad Nov 1 '17 at 9:22
  • $\begingroup$ The idea is that each set (document) should be considered "as a whole" when learning the model. Would adding a variable indicating which set the data is from be sufficient to do this? $\endgroup$ – gamesmad Nov 1 '17 at 9:25
  • $\begingroup$ Would it be possible to describe the setting in more detail? $\endgroup$ – Michael M Nov 1 '17 at 15:48
  • $\begingroup$ Of course, thanks for your help Michael. I have added a follow on question here. $\endgroup$ – gamesmad Nov 1 '17 at 16:52

This is a task. Classification is an enormous field.

You might want to start with Classification And Regression Trees (CARTs), which have the advantage of being easy to understand and explain. Plus, they are implemented in pretty much every ML or statistics package, such as R.

If you have thoroughly understood CARTs, you might want to look at Random Forests, which generalize CARTs and perform quite well across different classification tasks.

In any case, be sure to tell your system what kind of data you have. For instance, your word is numerically encoded, but I strongly suspect that it is actually categorical. Same for your target classification.

  • $\begingroup$ Thank you for your response Stephan. It has given me some good information to research my next question. Are CARTs appropriate for considering each of my proposed training sets "as a whole" as I mentioned in the comments above? Or am I missing the point? $\endgroup$ – gamesmad Nov 1 '17 at 9:26
  • $\begingroup$ Given that the documents are similar, one possibility would be to include a categorical predictor containing the document an observation comes from. However, that raises the question as to whether you would be predicting for a document you have already seen, versus for a completely new document. In the latter case, your CARTs will likely choke on the unobserved value in the "document" predictor. $\endgroup$ – Stephan Kolassa Nov 2 '17 at 9:33

In addition to the above answer by Stephan, I would recommend that you look at basic classification algorithms such as k-nearest neighbors first.

Also, it would be very useful to learn about best practices in how to train and evaluate your algorithms (cross-validation, grid search, etc). Best of luck!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.