Multi Categorical Features vs multiple Features for categories

Question

Say I am discretizing continuous data based on percentiles. (I realize this is generally frowned upon, but I am doing this for the sake of experiment) I am trying different percentiles, eg breaking the data into 10 categories, or 50 or 100. We will refer the amount of percentiles chosen as P.

Is there any difference in how the learning algorithm will function if I have one feature with P possible values, or if I have P feature, each of which is binary. I feel I am more drawn to the option that would give less dimensions (due to the fact that trying to visualize what 100 dimensions might look like may void my brain's warranty), but would like to know if there is any benefit to the higher dimension approach. Or is it very case specific?

(I am looking at logistic regression and SVM)

Answer 1

If you introduce $P$ binary categories without any relation, you will lose all information about ordering in the original continuous variable. Unless your sample size is extremely large, that will be very bad. If you introduce one categorical variable with $P$ categories, those should be ordered categories, else much the same will apply.

If your reason to try this is that you expect your continuous variable will act in a very non-linear fashion, then probably splines modeling will be a better alternative.

To get some more reasonable answer than this, you will need to include details about your real application.