16
$\begingroup$

Is it a separate type of distribution (EX: Binomial,bernoulli, Multinomial) or any distribution that can be represented this way? Can someone elaborate with a simple example

$\endgroup$

2 Answers 2

21
$\begingroup$

The categorical distribution is the generalization of the Bernoulli distribution to a fixed number $2 \le k$ of outcomes.

Equivalently, it is the special case of the multinomial distribution where the number of "choices" $n$ is fixed at one.

Therefore, it has pdf:

$$\prod_{i=1}^k p_i^{x_i} \qquad\text{(where $0\le p_i$ and $\sum_i p_i = 1$)}$$ over the support $$x_i \in \{0,1\}$$ where $$n \triangleq \sum_{i=1}^k x_i = 1.$$

In summary, Bernoulli has $k=2, n=1$, binomial has $k=2, n\ge 1$, multinomial has $k\ge2, n\ge1$, and categorical has $k\ge2, n=1$.

$\endgroup$
2
  • $\begingroup$ is that necessary, xi=0,1. Cant it be more than that. $\endgroup$
    – subha
    Aug 27, 2014 at 22:21
  • $\begingroup$ @subha: My understanding of categorical distribution has it that way. For multinomial and binomial, of course it can be. $\endgroup$
    – Neil G
    Aug 28, 2014 at 3:54
7
$\begingroup$

Categorical variables have finite sets of discrete values. Examples include sex (male/female), country, planet, etc. Contrast this with continuous variables, which can take an infinite number of different values. Examples include weight, longitude, distance, etc.

Note that similar information can sometimes be expressed in categorical and continuous ways; e.g., planet = earth could be expressed as distance to sun = 1 astronomical unit ≈ 150 million kilometers. However, there's not really any way to express 200 million kilometers from the sun in terms of planets, because there's no planet there (Mars is 228 million km from the sun). Same for 201 million km, 202, etc. All you could say about these distances in terms of planets is planet = none; you couldn't say planet = 4/3×earth or .88×Mars, because there's no meaningful way to multiply a planet or any other categorical variable. In terms of planets, these distances would be indistinguishable, but of course they make sense as distinct distances from the sun when expressed as such – as a continuous variable.

One can also express continuous variables with arbitrary precision (e.g., one astronomical unit is 149,597,871 km, not exactly 150 million km). Conversely, there is no way to express planet = earth more precisely; Earth is exactly earth, no more nor less. Furthermore, it would not make sense to say any other planet is "more" or "less" than Earth if planet is a nominal variable. It could be coded as an ordered (ordinal) variable though – planets are ordered in terms of distance to the sun, volume, number of moons, etc. These numbers are all continuous in their own terms (or at least counts, which are discrete but not categorical), but not in terms of planets. E.g., if planets are ordered by distance from the sun or by number of moons, mars > earth > venus. If planets are ordered by volume, earth > venus > mars. It is not necessary to order categorical variables, and maybe some cannot be ordered, but adding order does not make them any less categorical.

As Wikipedia says, categorical distributions are generalizations of the Bernoulli distribution to more than two possible values (the Bernoulli distribution is strictly binary). The Bernoulli distribution is also a special case of the binomial distribution, but I wouldn't call the binomial distribution categorical (it's discrete, but a count variable, so distances between values are defined). Multinomial distributions may be conflated with categorical distributions, but Wikipedia cautions against this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.