0
$\begingroup$

I want discretize my attributes according to the class quality which is the output variable. quality ranges from 1 to 10 in my data set. I think it'd be nice to have three quality categories: low, med, high. I defined low as 1-3, med as 4-6 and high as 7-10. However, the distribution of those categories is as follows:

enter image description here

There're very few instances with low quality. What would be the best approach to deal with those values? Should I discard them altogether, divide quality class into only 2 categories: not-high and high or pursue another approach?

EDIT: this is the dataset I'm analyzing: http://archive.ics.uci.edu/ml/datasets/Wine+Quality.

$\endgroup$
  • 1
    $\begingroup$ Not content with throwing away information, you want to throw away even more. I want to ask Why. Please try this question. I have information on people's heights. Not many people are more than 2 metres tall, so should I just lump those in with shorter people? There could be a rationale for using a coarser classification, but unless you tell us what this is, this is just a proposal to degrade data. $\endgroup$ – Nick Cox Mar 31 at 10:43
4
$\begingroup$

The best approach is to think again and not do this. Frank Harrell wrote, of categorizing continuous variables, "Nothing could be more disastrous" (Regression Modeling Strategies, 2nd ed p. 19).

I know you start off with a discrete scale, but the principle is the same.

I also recently put up an article on Medium about this, using graphs to show just how bad it is.

What should you do instead? That depends on what your goal is, what data you have, how it was collected and so on. Is this "Quality" going to be a dependent variable, an independent variable or what? Or are you just trying to describe the level of quality?

For instance, one place where quality ratings come into play are ratings of services (the little cards you sometimes get in hotels or restaurants or some other places). These are not used to measure satisfaction because the people who fill them out are not a random sample - they are more likely to be either very satisfied or very dissatisfied. Here, you would use these to identify problems or areas that are superlative.

If quality is a DV, then you might start with ordinal logistic regression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.