2
$\begingroup$

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) for a particular person. The purchases can range from zero to very many (no specific upper limit). Some of the features can be numeric and some categorical. The numeric features present a bigger problem because a categorical feature can always be coded with an extra category of "it doesn't exist" but what is the numberical value for "it doesn't exist"? Even a method that has sparse features don't completely solve this problem, because there still is the assumption of some fixed maximum number of features, but the real number can be arbitrary (maybe even greater in test set/new data than in training set).

It is possible to add a boolean feature for "the n-th feature exists" but this seems like a bad way to encode that problem (and does not work great on real data, I think mainly because the numeric non-existent values are not ignored even if the corresponding boolean "this feature exists" is false). It is also possible to use a random selection of a fixed number of features, or the top-n features using some ordering, etc but all these methods discard valuable information.

Are there any classifiers geared specifically to dealing with a variable number of features?

$\endgroup$

1 Answer 1

2
$\begingroup$

A general and short answer: Random Forest. It is well suited for such problem, because if any tree in the forest is using a non-existing variable, then this tree is just discarded.

I am also wondering: how your data are stored? Is it a data frame with NAs? or what?

$\endgroup$
1
  • $\begingroup$ @lanehook: Thanks! Good idea, I am already using RF. The data is stored in two csv files: id, feature (the id may occur many times) and id, class. In python it would be (id, class, [feature, feature, ...]). $\endgroup$
    – Alex I
    Apr 11, 2015 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.