8
$\begingroup$

I wonder if categorical data by definition can only take finitely or countably infinitely many values? And no more i.e. not uncountably many values?

Related question: is the distribution of a categorical variable always a discrete distribution or a continuous distribution?

$\endgroup$
4
  • $\begingroup$ What do you mean by "Countably infinite"? Using the definition from here: mathworld.wolfram.com/CountablyInfinite.html I can't actually conceive of an applied example. Do you just mean a variable that can range toward + or - infinity? $\endgroup$
    – Fomite
    Oct 15, 2011 at 7:35
  • $\begingroup$ No. I mean the cardinality of the set of the values which the categorical variable for the categorical data can take. $\endgroup$
    – Tim
    Oct 15, 2011 at 8:03
  • $\begingroup$ Not sure that added clarity ;) I'm going to take a swing anyway. $\endgroup$
    – Fomite
    Oct 15, 2011 at 8:05
  • $\begingroup$ I believe that the question should be rephrased can there be finite levels (or countably infinite) for a categorical variable. Categorical data can include infinite values - there's no reason to assume that an observation has a given cardinality. $\endgroup$
    – Iterator
    Oct 16, 2011 at 22:22

3 Answers 3

5
$\begingroup$

"Categorical" is not a well-defined mathematical term, so to answer this question we have to look to how this word is intended to be used. It is employed in contrast to "ordinal," "interval," and "ratio." One way to understand the primary distinctions is in terms of the groups of allowable re-expressions of the values. In the case of the latter three, there is an order that must preserved, whence all re-expressions must be monotonic (order preserving). For categorical variables, any bijection (including permutations) is ok:

Beyond that, anything goes with the nominal [categorical] scale.

(Stevens, quoted in the Wikipedia article.)

Another concept of "categorical" is that each outcome must be distinguishable from every other. This strongly suggests that any probability measure must be totally discrete: that is, all subsets are measurable, implying that each category will have its own well-defined probability. (This is not the case for continuous distributions.)

This would seem to indicate that the number of categories should be finite or at most countable, but that is not evident in the literature. For instance, an archetypal example of a categorical variable is a set of names. The set of all possible names on any finite alphabet is countable but not finite. It is therefore useful to allow countably infinite sets to be categorical. For example, if we are studying names given to babies, it is convenient to let the sample space consist of all possible names (rather than all names that we know of).

A slightly less realistic, but still conceivable, example of a categorical variable would be one that uses real numbers for names. In effect, such a variable would ignore all the usual mathematical structure on this set. I don't see any problem with such a usage, but it's worth observing that the axioms of probability imply that any probability distribution valid in this context would (a) assign a non-negative value to each real number and (b) would assign a non-zero value to at most a countable infinity of the reals.

One application involving an uncountable sample space that supports categorical random variables of infinite, even uncountable, support is the study of random graphs. To understand the rate of growth of some property of graphs, we would want to contemplate graphs on 0, 1, ..., $n$, ... nodes, so it's convenient to allow graphs to have countably many nodes. Random variables defined on this set can have various types. For instance, the mean vertex degree (if finite) could be considered of ratio type; the total vertex degree could be considered of ordinal type (and, therefore--by forgetting the ordering--is a nice example of a countable discrete variable). If we also allow a graph to have arbitrarily many edges and are interested in, say, its connected components, then we would have a naturally occurring category that is uncountable (because each connected component determines the subset of nodes it contains and there are uncountably many subsets of a countable set).

To summarize, it is reasonable to allow categorical values to attain an uncountable infinity of possible values, while recognizing that at most a countable number of them could ever have positive probabilities. This must be a discrete distribution, because all subsets are measurable, which is not the case for continuous distributions.

$\endgroup$
5
  • 1
    $\begingroup$ "it is convenient to let the sample space consist of all possible names (rather than all names that we know of)" - how is that convenient? $\endgroup$
    – Fomite
    Oct 16, 2011 at 21:30
  • $\begingroup$ I'm not sure why you say that "categorical" is not well-defined mathematically. I believe that you may need to clarify the meaning of category, level, and data. These terms must be defined mathematically for mathematical statistics to be relevant. :) Nonetheless, it's quite an interesting answer. $\endgroup$
    – Iterator
    Oct 16, 2011 at 22:29
  • $\begingroup$ @Iterator I have no disagreement with you and appreciate your remarks. To clarify where I'm coming from, it appears to me that the scale "categorical, nominal, interval, ratio" is not a mathematical one, but rather is a qualitative description. One could indeed come up with mathematical definitions that capture most of the concept of "categorical," but that might even be counterproductive. I do not view this hierarchy as being any part of mathematical statistics, and in fact find it to be useless except as a suggestive guide in some applications. $\endgroup$
    – whuber
    Oct 16, 2011 at 23:46
  • $\begingroup$ @whuber It sort of matters for issues like constructing hypothesis tests, and then for applications such as estimating necessary sample sizes. :) In any case, the OP was vague. Categorical is an adjective, which noun it modifies is the issue to address. Whether that noun is countable or describes something countable is the source of ambiguity. I am contemplating editing the question to reflect something that's actually answerable, because the vagueness yields an unanswerable question. $\endgroup$
    – Iterator
    Oct 16, 2011 at 23:56
  • $\begingroup$ @Iterator I think you're on to something here. I have been wondering whether "categorical" refers to a sample space or to a random variable or to neither. In his original article, Stevens addresses scales of measurement. As such, it would seem he was interested in an additional property of a random variable. I would characterize this extra property in terms of the group of transformations of $\mathbb{R}$ that would be considered inconsequential for the data analysis. However, I'm unaware of any published definition of this sort. $\endgroup$
    – whuber
    Oct 17, 2011 at 13:36
3
$\begingroup$

Alright, here's my attempt at an answer, from my (admittedly imperfect) understanding of your question.

"Categorical data" is something of a fuzzy and problematic term to begin with. Similar to the "I know it when I see it" definition of obscene material. There are some very clear cases of categorical data, where the values of a variable fall into a small number of clearly defined categories.

Beyond that, there be dragons.

At some point you get enough categories that your "categorical" variable could likely be treated as a continuous variable. Or alternately, using either subject matter knowledge or a description of a distribution, you can break a continuous variable up into categorical chunks and treat it as categorical.

So there's really two answers to your question:

Theory Answer: No. You could have infinitely many categories, but for some reason decide not to call it a continuous variable. If you allow for decimal-based categories of an utterly unbounded variable, there's no reason I can see it wouldn't be be uncountably infinite.

I'm not sure how often this ends up coming up. In my experience at least, fairly rarely.

Applied Answer: The cardinality of most things that would ever reasonably be called categorical data have a cardinality vastly below the cardinality of N. As noted above, there are exceptions, often subject to vague judgement calls.

$\endgroup$
19
  • 1
    $\begingroup$ Thanks! So the silly answer is that the cardinality can be any, while the applied answer is that mostly it is finite? $\endgroup$
    – Tim
    Oct 15, 2011 at 8:20
  • 2
    $\begingroup$ @PeterFlom: Cantor showed that the real numbers are uncountable. :) $\endgroup$
    – cardinal
    Oct 16, 2011 at 2:05
  • 3
    $\begingroup$ @EpiGrad: I don't consider your "silly" answer quite so silly. There is no particular reason I see that an infinite number of categories should somehow be equated with some form of continuous random variable. Indeed, the difference seems to hinge on whether or not the variable can be represented in a meaningful, natural, and unambiguous way as a real number. $\endgroup$
    – cardinal
    Oct 16, 2011 at 2:11
  • 2
    $\begingroup$ What would "vastly below the cardinality of N" mean? Is this just a long way of saying "finite"? If so, that would rule out many interesting applications involving categorical variables. $\endgroup$
    – whuber
    Oct 16, 2011 at 15:36
  • 1
    $\begingroup$ Hi @cardinal yes, I know Cantor showed the reals are uncountable. But is there any set of categories that needs more than the rationals, which are countable? I have trouble imagining a set of categories that requires use of irrationals. $\endgroup$
    – Peter Flom
    Oct 17, 2011 at 11:07
1
$\begingroup$

Categorical data is discrete - otherwise it would be hard to assign categories to the data.

My take is this: the natural numbers are discrete and thus categorical. They are also ordinal and interval data, but also categorical. Since the natural numbers are countably infinite we see that there are categorical variables that can take countably infinite values. That does not mean, though, that this applies for all categorical variables.

$\endgroup$
1
  • $\begingroup$ I believe that the usage is ambiguous. Categorical variables have levels. Data is merely observations of a categorical variable, with some specified level. One could abuse the notion of levels to produce infinitely many. A level itself could, say, count the infinities encountered in a data set (e.g. $X/Y$ where both are drawn from the two element set ${0,1}$. I do not disagree with your answer, but suggest clarifying to address the issue of levels (or categories). $\endgroup$
    – Iterator
    Oct 16, 2011 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.