I am running multiple logistic regression to understand feature importance. One of the independent variables has a distribution (histogram) that looks like:
| Value | Count |
|---|---|
| 0 | 4000 |
| 1 | 25 |
| 2 | 10 |
| 3 | 10 |
| 4 | 1 |
There is, obviously, a long tail here. My gut instinct is to bucket this long tail into a single value such that the distribution becomes a binary categorical variable:
| Value | Count |
|---|---|
| 0 | 4000 |
| Not_0 | 178 |
Is this the right instinct? I wouldn't want my coefficient from the regression to just be noise, which I anticipate the non-bucketed distribution would be susceptible to. Maybe I am misunderstand logistic regression though.
Another choice might be not using this predictor at all due to the class imbalance.
