In answering this question on discrete and continuous data I glibly asserted that it rarely makes sense to treat categorical data as continuous.
On the face of it that seems self-evident, but intuition is often a poor guide for statistics, or at least mine is. So now I'm wondering: is it true? Or are there established analyses for which a transform from categorical data to some continuum is actually useful? Would it make a difference if the data were ordinal?
I will assume that a "categorical" variable actually stands for an ordinal variable; otherwise it doesn't make much sense to treat it as a continuous one, unless it's a binary variable (coded 0/1) as pointed by @Rob. Then, I would say that the problem is not that much the way we treat the variable, although many models for categorical data analysis have been developed so far--see e.g., The analysis of ordered categorical data: An overview and a survey of recent developments from Liu and Agresti--, than the underlying measurement scale we assume. My response will focus on this second point, although I will first briefly discuss the assignment of numerical scores to variable categories or levels.
By using a simple numerical recoding of an ordinal variable, you are assuming that the variable has interval properties (in the sense of the classification given by Stevens, 1946). From a measurement theory perspective (in psychology), this may often be a too strong assumption, but for basic study (i.e. where a single item is used to express one's opinion about a daily activity with clear wording) any monotone scores should give comparable results. Cochran (1954) already pointed that
any set of scores gives a valid test, provided that they are constructed without consulting the results of the experiment. If the set of scores is poor, in that it badly distorts a numerical scale that really does underlie the ordered classification, the test will not be sensitive. The scores should therefore embody the best insight available about the way in which the classification was constructed and used. (p. 436)
(Many thanks to @whuber for reminding me about this throughout one of his comments, which led me to re-read Agresti's book, from which this citation comes.)
Actually, several tests treat implicitly such variables as interval scales: for example, the $M^2$ statistic for testing a linear trend (as an alternative to simple independence) is based on a correlational approach ($M^2=(n-1)r^2$, Agresti, 2002, p. 87).
Well, you can also decide to recode your variable on an irregular range, or aggregate some of its levels, but in this case strong imbalance between recoded categories may distort statistical tests, e.g. the aforementioned trend test. A nice alternative for assigning distance between categories was already proposed by @Jeromy, namely optimal scaling.
Now, let's discuss the second point I made, that of the underlying measurement model. I'm always hesitating about adding the "psychometrics" tag when I see this kind of question, because the construction and analysis of measurement scales come under Psychometric Theory (Nunnally and Bernstein, 1994, for a neat overview). I will not dwell on all the models that are actually headed under the Item Response Theory, and I kindly refer the interested reader to I. Partchev's tutorial, A visual guide to item response theory, for a gentle introduction to IRT, and to references (5-8) listed at the end for possible IRT taxonomies. Very briefly, the idea is that rather than assigning arbitrary distances between variable categories, you assume a latent scale and estimate their location on that continuum, together with individuals' ability or liability. A simple example is worth much mathematical notation, so let's consider the following item (coming from the EORTC QLQ-C30 health-related quality of life questionnaire):
Did you worry?
which is coded on a four-point scale, ranging from "Not at all" to "Very much". Raw scores are computed by assigning a score of 1 to 4. Scores on items belonging to the same scale can then be added together to yield a so-called scale score, which denotes one's rank on the underlying construct (here, a mental health component). Such summated scale scores are very practical because of scoring easiness (for the practitioner or nurse), but they are nothing more than a discrete (ordered) scale.
We can also consider that the probability of endorsing a given response category obeys some kind of a logistic model, as described in I. Partchev's tutorial, referred above. Basically, the idea is that of a kind of threshold model (which lead to equivalent formulation in terms of the proportional or cumulative odds models) and we model the odds of being in one response category rather the preceding one or the odds of scoring above a certain category, conditional on subjects' location on the latent trait. In addition, we may impose that response categories are equally spaced on the latent scale (this is the Rating Scale model)--which is the way we do by assigning regularly spaced numerical scores-- or not (this is the Partial Credit model).
Clearly, we are not adding very much to Classical Test Theory, where ordinal variable are treated as numerical ones. However, we introduce a probabilistic model, where we assume a continuous scale (with interval properties) and where specific errors of measurement can be accounted for, and we can plug these factorial scores in any regression model.
References
If there are only two categories, then transforming them to (0,1) makes sense. In fact, this is commonly done where the resulting dummy variable is used in regression models.
If there are more than two categories, then I think it only makes sense if the data are ordinal, and then only in very specific circumstances. For example, if I am doing regression and fit a nonparametric nonlinear function to the ordinal-cum-numeric variable, I think that is ok. But if I use linear regression, then I am making very strong assumptions about the relative difference between consecutive values of the ordinal variable, and I'm usually reluctant to do that.
It is common practice to treat ordered categorical variables with many categories as continuous. Examples of this:
And by "treating as continuous" I mean including the variable in a model that assumes a continuous random variable (e.g., as a dependent variable in a linear regression). I suppose the issue is how many scale points are required for this to be a reasonable simplifying assumption.
A few other thoughts:
A very simple example often overlooked that should lie within the experience of many readers concerns the marks or grades given to academic work. Often marks for individual assignments are in essence judgement-based ordinal measurements, even when as a matter of convention they are given as (say) percent marks or marks on a scale with maximum 5 (possibly with decimal points too). That is, a teacher may read through an essay or dissertation or thesis or paper and decide that it deserves 42%, or 4, or whatever. Even when marks are based on a detailed assessment scheme the scale is at root some distance from an interval or ratio measurement scale.
But then many institutions take the view that if you have enough of these marks or grades it is perfectly reasonable to average them (grade-point average, etc.) and even to analyse them in more detail. So at some point the ordinal measurements morph into a summary scale that is treated as if it were continuous.
Connoisseurs of irony will note that statistical courses in many Departments or Schools often teach that this is at best dubious and at worst wrong, all the while it is implemented as a University-wide procedure.
In an analysis of ranking by frequency, as with a Pareto chart and associated values (eg how many categories make up the top 80% of product faults)
I'm going to make the argument that treating a truly categorical, non-ordinal variable as continuous can sometimes make sense.
If you are building decision trees based on large datasets, it may be costly in terms of processing power and memory to convert categorical variables into dummy variables. Furthermore, some models (e.g. randomForest in R) cannot handle categorical variables with many levels.
In these cases, a tree-based model should be able to identify extremely important categories, EVEN IF they are coded as a continuous variable. A contrived example:
set.seed(42)
library(caret)
n <- 10000
a <- sample(1:100, n, replace=TRUE)
b <- sample(1:100, n, replace=TRUE)
e <- runif(n)
y <- 2*a + 1000*(b==7) + 500*(b==42) + 1000*e
dat1 <- data.frame(y, a, b)
dat2 <- data.frame(y, a, b=factor(b))
y is a continuous variable, a is a continuous variable, and b is a categorical variable. However, in dat1 b is treated as continuous.
Fitting a decision tree to these 2 datasets, we find that dat1 is slightly worse than dat2:
model1 <- train(y~., dat1, method='rpart')
model2 <- train(y~., dat2, method='rpart')
> min(model1$results$RMSE)
[1] 302.0428
> min(model2$results$RMSE)
[1] 294.1411
If you look at the 2 models, you will find that they are very similar, but model1 misses the importance of b==42:
> model1$finalModel
n= 10000
node), split, n, deviance, yval
* denotes terminal node
1) root 10000 988408000 614.0377
2) a< 42.5 4206 407731400 553.5374 *
3) a>=42.5 5794 554105700 657.9563
6) b>=7.5 5376 468539000 649.2613 *
7) b< 7.5 418 79932820 769.7852
14) b< 6.5 365 29980450 644.6897 *
15) b>=6.5 53 4904253 1631.2920 *
> model2$finalModel
n= 10000
node), split, n, deviance, yval
* denotes terminal node
1) root 10000 988408000 614.0377
2) b7< 0.5 9906 889387900 604.7904
4) a< 42.5 4165 364209500 543.8927 *
5) a>=42.5 5741 498526600 648.9707
10) b42< 0.5 5679 478456300 643.7210 *
11) b42>=0.5 62 5578230 1129.8230 *
3) b7>=0.5 94 8903490 1588.5500 *
However, model1 runs in about 1/10 of the time of model2:
> model1$times$everything
user system elapsed
4.881 0.169 5.058
> model2$times$everything
user system elapsed
45.060 3.016 48.066
You can of course tweak the parameters of the problem to find situations in which dat2 far outperforms dat1, or dat1 slightly outperforms dat2.
I am not advocating generally treating categorical variables as continuous, but I have found situations where doing so has greatly reduced the time it takes to fit my models, without decreasing their predictive accuracy.
A very nice summary of this topic can be found here.
"When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under sub-optimal conditions."
Mijke Rhemtulla, Patricia É. Brosseau-Liard, and Victoria Savalei
They investigate about 60 pages' worth of methods for doing so and provide insights as to when it's useful to do, which approach to take, and what the strengths and weaknesses are of each approach to fit your specific situation. They don't cover all of them (as I'm learning there seems to be a limitless amount), but the ones they do cover they cover well.
There is another case when it makes sense: when the data is sampled from continuous data (for example through an analogue-to-digital converter). For older instruments the ADCs would often be 10-bit, giving what is nominally 1024-category ordinal data, but can for most purposes be treated as real (though there will be some artifacts for values near the low end of the scale). Today ADCs are more commonly 16 or 24-bit. By the time you're talking 65536 or 16777216 "categories", you really have no trouble treating the data as continuous.
