57
$\begingroup$

In answering this question on discrete and continuous data I glibly asserted that it rarely makes sense to treat categorical data as continuous.

On the face of it that seems self-evident, but intuition is often a poor guide for statistics, or at least mine is. So now I'm wondering: is it true? Or are there established analyses for which a transform from categorical data to some continuum is actually useful? Would it make a difference if the data were ordinal?

$\endgroup$
  • 20
    $\begingroup$ This question and its responses remind us of how crude and limited this antiquated division of variables into categorical-ordinal-interval-ratio really is. It can guide the statistically naive, but for the thoughtful or experienced analyst it's a hindrance, an obstacle in the way of expressing variables in ways that are appropriate for the data and the decisions to be made with them. Someone working from this latter point of view will freely move between categorical and "continuous" data representations; for them, this question cannot even arise! Instead, we should ask: how does it help? $\endgroup$ – whuber Nov 3 '10 at 20:29
  • $\begingroup$ @whuber (+1) At the very least, it seems difficult to optimize measurement reliability and diagnostic accuracy at the same time. $\endgroup$ – chl Nov 3 '10 at 21:02
34
$\begingroup$

I will assume that a "categorical" variable actually stands for an ordinal variable; otherwise it doesn't make much sense to treat it as a continuous one, unless it's a binary variable (coded 0/1) as pointed by @Rob. Then, I would say that the problem is not that much the way we treat the variable, although many models for categorical data analysis have been developed so far--see e.g., The analysis of ordered categorical data: An overview and a survey of recent developments from Liu and Agresti--, than the underlying measurement scale we assume. My response will focus on this second point, although I will first briefly discuss the assignment of numerical scores to variable categories or levels.

By using a simple numerical recoding of an ordinal variable, you are assuming that the variable has interval properties (in the sense of the classification given by Stevens, 1946). From a measurement theory perspective (in psychology), this may often be a too strong assumption, but for basic study (i.e. where a single item is used to express one's opinion about a daily activity with clear wording) any monotone scores should give comparable results. Cochran (1954) already pointed that

any set of scores gives a valid test, provided that they are constructed without consulting the results of the experiment. If the set of scores is poor, in that it badly distorts a numerical scale that really does underlie the ordered classification, the test will not be sensitive. The scores should therefore embody the best insight available about the way in which the classification was constructed and used. (p. 436)

(Many thanks to @whuber for reminding me about this throughout one of his comments, which led me to re-read Agresti's book, from which this citation comes.)

Actually, several tests treat implicitly such variables as interval scales: for example, the $M^2$ statistic for testing a linear trend (as an alternative to simple independence) is based on a correlational approach ($M^2=(n-1)r^2$, Agresti, 2002, p. 87).

Well, you can also decide to recode your variable on an irregular range, or aggregate some of its levels, but in this case strong imbalance between recoded categories may distort statistical tests, e.g. the aforementioned trend test. A nice alternative for assigning distance between categories was already proposed by @Jeromy, namely optimal scaling.

Now, let's discuss the second point I made, that of the underlying measurement model. I'm always hesitating about adding the "psychometrics" tag when I see this kind of question, because the construction and analysis of measurement scales come under Psychometric Theory (Nunnally and Bernstein, 1994, for a neat overview). I will not dwell on all the models that are actually headed under the Item Response Theory, and I kindly refer the interested reader to I. Partchev's tutorial, A visual guide to item response theory, for a gentle introduction to IRT, and to references (5-8) listed at the end for possible IRT taxonomies. Very briefly, the idea is that rather than assigning arbitrary distances between variable categories, you assume a latent scale and estimate their location on that continuum, together with individuals' ability or liability. A simple example is worth much mathematical notation, so let's consider the following item (coming from the EORTC QLQ-C30 health-related quality of life questionnaire):

Did you worry?

which is coded on a four-point scale, ranging from "Not at all" to "Very much". Raw scores are computed by assigning a score of 1 to 4. Scores on items belonging to the same scale can then be added together to yield a so-called scale score, which denotes one's rank on the underlying construct (here, a mental health component). Such summated scale scores are very practical because of scoring easiness (for the practitioner or nurse), but they are nothing more than a discrete (ordered) scale.

We can also consider that the probability of endorsing a given response category obeys some kind of a logistic model, as described in I. Partchev's tutorial, referred above. Basically, the idea is that of a kind of threshold model (which lead to equivalent formulation in terms of the proportional or cumulative odds models) and we model the odds of being in one response category rather the preceding one or the odds of scoring above a certain category, conditional on subjects' location on the latent trait. In addition, we may impose that response categories are equally spaced on the latent scale (this is the Rating Scale model)--which is the way we do by assigning regularly spaced numerical scores-- or not (this is the Partial Credit model).

Clearly, we are not adding very much to Classical Test Theory, where ordinal variable are treated as numerical ones. However, we introduce a probabilistic model, where we assume a continuous scale (with interval properties) and where specific errors of measurement can be accounted for, and we can plug these factorial scores in any regression model.

References

  1. S S Stevens. On the theory of scales of measurement. Science, 103: 677-680, 1946.
  2. W G Cochran. Some methods of strengthening the common $\chi^2$ tests. Biometrics, 10: 417-451, 1954.
  3. J Nunnally and I Bernstein. Psychometric Theory. McGraw-Hill, 1994
  4. Alan Agresti. Categorical Data Analysis. Wiley, 1990.
  5. C R Rao and S Sinharay, editors. Handbook of Statistics, Vol. 26: Psychometrics. Elsevier Science B.V., The Netherlands, 2007.
  6. A Boomsma, M A J van Duijn, and T A B Snijders. Essays on Item Response Theory. Springer, 2001.
  7. D Thissen and L Steinberg. A taxonomy of item response models. Psychometrika, 51(4): 567–577, 1986.
  8. P Mair and R Hatzinger. Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R. Journal of Statistical Software, 20(9), 2007.
$\endgroup$
19
$\begingroup$

If there are only two categories, then transforming them to (0,1) makes sense. In fact, this is commonly done where the resulting dummy variable is used in regression models.

If there are more than two categories, then I think it only makes sense if the data are ordinal, and then only in very specific circumstances. For example, if I am doing regression and fit a nonparametric nonlinear function to the ordinal-cum-numeric variable, I think that is ok. But if I use linear regression, then I am making very strong assumptions about the relative difference between consecutive values of the ordinal variable, and I'm usually reluctant to do that.

$\endgroup$
  • 1
    $\begingroup$ "[T]hen I am making very strong assumptions about the relative difference between consecutive values of the ordinal variable." I think this is the key point, really. i.e. how strongly can you argue that the difference between groups 1 and 2 is comparable to that between 2 and 3? $\endgroup$ – Freya Harrison Jul 23 '10 at 10:43
  • $\begingroup$ I think you should make some assumption about how the continuous variable should be distributed and then try to fit this "psudohistogram" of each categorical variable frequency (I mean find bin widths which will transform it into a fitted histogram). Still, I'm not an expert in this field, its a fast&dirty idea. $\endgroup$ – user88 Jul 23 '10 at 13:56
  • $\begingroup$ Recasting binary categories as {0,1} makes sense, but turning that into a continuous [0,1] interval seems like a bit of a leap. On the broader front, I'm totally with your reluctance to weight ordinals equally unless there are powerful arguments from the model. $\endgroup$ – walkytalky Jul 23 '10 at 15:29
18
$\begingroup$

It is common practice to treat ordered categorical variables with many categories as continuous. Examples of this:

  • Number of items correct on a 100 item test
  • A summated psychological scale (e.g., that is the mean of 10 items each on a five point scale)

And by "treating as continuous" I mean including the variable in a model that assumes a continuous random variable (e.g., as a dependent variable in a linear regression). I suppose the issue is how many scale points are required for this to be a reasonable simplifying assumption.

A few other thoughts:

  • Polychoric correlations attempt to model the relationship between two ordinal variables in terms of assumed latent continuous variables.
  • Optimal scaling allows you to develop models where the scaling of a categorical variable is developed in a data driven way whilst respecting whatever scale constraints you impose (e.g., ordinality). For a good introduction see De Leeuw and Mair (2009)

References

  • De Leeuw, J., & Mair, P. (2009). Gifi methods for optimal scaling in R: The package homals. Journal of Statistical Software, forthcoming, 1-30. PDF
$\endgroup$
7
$\begingroup$

A very simple example often overlooked that should lie within the experience of many readers concerns the marks or grades given to academic work. Often marks for individual assignments are in essence judgement-based ordinal measurements, even when as a matter of convention they are given as (say) percent marks or marks on a scale with maximum 5 (possibly with decimal points too). That is, a teacher may read through an essay or dissertation or thesis or paper and decide that it deserves 42%, or 4, or whatever. Even when marks are based on a detailed assessment scheme the scale is at root some distance from an interval or ratio measurement scale.

But then many institutions take the view that if you have enough of these marks or grades it is perfectly reasonable to average them (grade-point average, etc.) and even to analyse them in more detail. So at some point the ordinal measurements morph into a summary scale that is treated as if it were continuous.

Connoisseurs of irony will note that statistical courses in many Departments or Schools often teach that this is at best dubious and at worst wrong, all the while it is implemented as a University-wide procedure.

$\endgroup$
5
$\begingroup$

In an analysis of ranking by frequency, as with a Pareto chart and associated values (eg how many categories make up the top 80% of product faults)

$\endgroup$
  • 5
    $\begingroup$ Important point, and it can be extended: Many models for ordinal data hinge on the idea that it's not the ordinal data but their cumulative probabilities that can be modelled. $\endgroup$ – Nick Cox Aug 12 '13 at 13:47
4
$\begingroup$

I'm going to make the argument that treating a truly categorical, non-ordinal variable as continuous can sometimes make sense.

If you are building decision trees based on large datasets, it may be costly in terms of processing power and memory to convert categorical variables into dummy variables. Furthermore, some models (e.g. randomForest in R) cannot handle categorical variables with many levels.

In these cases, a tree-based model should be able to identify extremely important categories, EVEN IF they are coded as a continuous variable. A contrived example:

set.seed(42)
library(caret)
n <- 10000
a <- sample(1:100, n, replace=TRUE)
b <- sample(1:100, n, replace=TRUE)
e <- runif(n)
y <- 2*a + 1000*(b==7) + 500*(b==42) + 1000*e
dat1 <- data.frame(y, a, b)
dat2 <- data.frame(y, a, b=factor(b))

y is a continuous variable, a is a continuous variable, and b is a categorical variable. However, in dat1 b is treated as continuous.

Fitting a decision tree to these 2 datasets, we find that dat1 is slightly worse than dat2:

model1 <- train(y~., dat1, method='rpart')
model2 <- train(y~., dat2, method='rpart')
> min(model1$results$RMSE)
[1] 302.0428
> min(model2$results$RMSE)
[1] 294.1411

If you look at the 2 models, you will find that they are very similar, but model1 misses the importance of b==42:

> model1$finalModel
n= 10000 

node), split, n, deviance, yval
      * denotes terminal node

 1) root 10000 988408000  614.0377  
   2) a< 42.5 4206 407731400  553.5374 *
   3) a>=42.5 5794 554105700  657.9563  
     6) b>=7.5 5376 468539000  649.2613 *
     7) b< 7.5 418  79932820  769.7852  
      14) b< 6.5 365  29980450  644.6897 *
      15) b>=6.5 53   4904253 1631.2920 *
> model2$finalModel
n= 10000 

node), split, n, deviance, yval
      * denotes terminal node

 1) root 10000 988408000  614.0377  
   2) b7< 0.5 9906 889387900  604.7904  
     4) a< 42.5 4165 364209500  543.8927 *
     5) a>=42.5 5741 498526600  648.9707  
      10) b42< 0.5 5679 478456300  643.7210 *
      11) b42>=0.5 62   5578230 1129.8230 *
   3) b7>=0.5 94   8903490 1588.5500 *

However, model1 runs in about 1/10 of the time of model2:

> model1$times$everything
   user  system elapsed 
  4.881   0.169   5.058 
> model2$times$everything
   user  system elapsed 
 45.060   3.016  48.066 

You can of course tweak the parameters of the problem to find situations in which dat2 far outperforms dat1, or dat1 slightly outperforms dat2.

I am not advocating generally treating categorical variables as continuous, but I have found situations where doing so has greatly reduced the time it takes to fit my models, without decreasing their predictive accuracy.

$\endgroup$
3
$\begingroup$

A very nice summary of this topic can be found here:

mijkerhemtulla.socsci.uva.nl PDF

"When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under sub-optimal conditions."

Mijke Rhemtulla, Patricia É. Brosseau-Liard, and Victoria Savalei

They investigate about 60 pages' worth of methods for doing so and provide insights as to when it's useful to do, which approach to take, and what the strengths and weaknesses are of each approach to fit your specific situation. They don't cover all of them (as I'm learning there seems to be a limitless amount), but the ones they do cover they cover well.

$\endgroup$
2
$\begingroup$

There is another case when it makes sense: when the data is sampled from continuous data (for example through an analogue-to-digital converter). For older instruments the ADCs would often be 10-bit, giving what is nominally 1024-category ordinal data, but can for most purposes be treated as real (though there will be some artifacts for values near the low end of the scale). Today ADCs are more commonly 16 or 24-bit. By the time you're talking 65536 or 16777216 "categories", you really have no trouble treating the data as continuous.

$\endgroup$
  • $\begingroup$ I agree strongly with your bottom line, but arguably such data were never ordinal to start with, just discretised. Lousy treatments of nominal-ordinal-interval-ratio are to blame here for often not pointing that ordinal implies discrete, but not vice versa. A count is ordinal, but it is interval and ratio too. $\endgroup$ – Nick Cox Aug 12 '13 at 15:02
  • $\begingroup$ @Nick Ordinal implies discrete? Not necessarily. Continuous measures can be ordinal. For instance, physiological variables such as GSP or heart rate are continuous, but as measures of psychological variables such as anxiety or arousal they are only ordinal. The notion of ordinal vs interval really refers to the linearity of the function that relates the measure to what it is intended to measure. $\endgroup$ – Ray Koopman Aug 13 '13 at 6:31
  • $\begingroup$ That's an interesting remark, but once you get into that territory I don't see how you can classify heart rate at all without independent evidence of what anxiety really is and ultimately most variables regarded as proxies are thus unclassifiable. Would you take it all the way to refusing to use methods for interval or ratio data whenever you switch to regarding the measurement scale as only ordinal? I don't think the data behave differently because of what you intend to do with them; that's the nub of the issue for me. $\endgroup$ – Nick Cox Aug 13 '13 at 7:40
  • 1
    $\begingroup$ @Nick The question is whether the function that relates the measured value to the "true" value is sufficiently close to linear that treating it as such will not lead to wrong substantive conclusions, or must it be treated as only monotonic. There is usually little or no hard data on which to base the decision; it will almost always be a judgment call, about which informed intelligent people may have to agree to disagree. $\endgroup$ – Ray Koopman Aug 14 '13 at 2:56
  • 1
    $\begingroup$ I think the usual emphasis in discussions of measurement scales in statistics is on the mathematical properties of variables and what are the legitimate mathematical operations for each. That's contentious enough. Scientific concern with whether something measures what is supposed to I readily agree to be vitally important, but I see as a rather different area of debate. $\endgroup$ – Nick Cox Aug 14 '13 at 7:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.