4
$\begingroup$

Say I am discretizing continuous data based on percentiles. (I realize this is generally frowned upon, but I am doing this for the sake of experiment) I am trying different percentiles, eg breaking the data into 10 categories, or 50 or 100. We will refer the amount of percentiles chosen as P.

Is there any difference in how the learning algorithm will function if I have one feature with P possible values, or if I have P feature, each of which is binary. I feel I am more drawn to the option that would give less dimensions (due to the fact that trying to visualize what 100 dimensions might look like may void my brain's warranty), but would like to know if there is any benefit to the higher dimension approach. Or is it very case specific?

(I am looking at logistic regression and SVM)

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I would say it depends on the data; look at the histogram of your original matrix. If it is well distributed may be it does not matter. look into the variance of the data and see much variability each P has ! you don't want to have a lot of low variance features ... I agree with the answer that depends on your data ! you can also consider using a transformation function say, (log(x)) ... $\endgroup$
    – Areza
    Jan 26, 2015 at 13:17

1 Answer 1

Reset to default
1
$\begingroup$

If you introduce $P$ binary categories without any relation, you will lose all information about ordering in the original continuous variable. Unless your sample size is extremely large, that will be very bad. If you introduce one categorical variable with $P$ categories, those should be ordered categories, else much the same will apply.

If your reason to try this is that you expect your continuous variable will act in a very non-linear fashion, then probably splines modeling will be a better alternative.

To get some more reasonable answer than this, you will need to include details about your real application.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.