1
$\begingroup$

I am given a dataset with multiple features associated with each data point. I would like to select a subset of data points and find out what features best separate the selected subset from the remaining data points. I'm especially interested in interpretable, less black-boxy methods since I would want to know the features and, ideally, some specific information about what values fall into which subgroup.

What method would be most suitable for such a task? One thing I had in mind was to take those two groups, assign labels to them, run logistic regression and subsequently extract feature importances from them, but maybe there is a smarter solution. Also, the selected subgroup could be significantly smaller than the remaining group, what should I keep in mind when the groups are unbalanced, what could I do to mitigate this?

Please ask in the comments if some clarification is needed, and no, this is not a course assignment but a real challenge I face :).

$\endgroup$

2 Answers 2

0
$\begingroup$

You could probably build a decision tree after giving labels to the two classes. Then you could view how each example is going down the decision tree, and see how subgroups are formed after each feature division.[Check if you could get the information gain of splitting along each feature for the first node.Using this also you can judge what features are useful].

Ideally if you don't want to see how subgroups are being formed you could use Feature ranking output by Random Forest.

$\endgroup$
0
$\begingroup$

If you want to find a linear boundary, linear discriminant analysis (LDA) is a good choice Wikipedia page on 'Linear Discriminant Analysis'. Logistic regression would also probably yield a similar result. You can add polynomial features to either of these methods to find nonlinear boundaries, but interpreting the results could get more complicated.

I think your instinct to start with logistic regression and inspect the coefficients is not a bad idea.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.