Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Is this separate type of distribution (EX: Binomial,bernoulli, Multinomial) or any distribution can be represented by this way. Can someone elaborate with simple example

share|improve this question

2 Answers 2

up vote 2 down vote accepted

The categorical distribution is the generalization of the Bernoulli distribution to a fixed number $2 \le k$ of outcomes.

Equivalently, it is the special case of the multinomial distribution where the number of "choices" $n$ is fixed at one.

Therefore, it has pdf:

$$\prod_{i=1}^k p_i^{x_i} \qquad\text{(where $0\le p_i$ and $\sum_i p_i = 1$)}$$ over the support $$x_i \in \{0,1\}$$ where $$n \triangleq \sum_{i=1}^k x_i = 1.$$

In summary, bernoulli has $k=2, n=1$, binomial has $k=2, n\ge 1$, multinomial has $k\ge2, n\ge1$, and categorical has $k\ge2, n=1$.

share|improve this answer
    
is that necessary, xi=0,1. Cant it be more than that. –  subha Aug 27 at 22:21
    
@subha: My understanding of categorical distribution has it that way. For multinomial and binomial, of course it can be. –  Neil G Aug 28 at 3:54

Categorical variables have finite sets of discrete values. Examples include sex (male/female), country, planet, etc. Contrast this with continuous variables, which can take an infinite number of different values. Examples include weight, longitude, distance, etc.

Note that similar information can sometimes be expressed in categorical and continuous ways; e.g., planet = earth could be expressed as distance to sun = 1 astronomical unit ≈ 150 million kilometers. However, there's not really any way to express 200 million kilometers from the sun in terms of planets, because there's no planet there (Mars is 228 million km from the sun). Same for 201 million km, 202, etc. All you could say about these distances in terms of planets is planet = none; you couldn't say planet = 4/3×earth or .88×Mars, because there's no meaningful way to multiply a planet or any other categorical variable. In terms of planets, these distances would be indistinguishable, but of course they make sense as distinct distances from the sun when expressed as such – as a continuous variable.

One can also express continuous variables with arbitrary precision (e.g., one astronomical unit is 149,597,871 km, not exactly 150 million km). Conversely, there is no way to express planet = earth more precisely; Earth is exactly earth, no more nor less. Furthermore, it would not make sense to say any other planet is "more" or "less" than Earth if planet is a nominal variable. It could be coded as an ordered (ordinal) variable though – planets are ordered in terms of distance to the sun, volume, number of moons, etc. These numbers are all continuous in their own terms (or at least counts, which are discrete but not categorical), but not in terms of planets. E.g., if planets are ordered by distance from the sun or by number of moons, mars > earth > venus. If planets are ordered by volume, earth > venus > mars. It is not necessary to order categorical variables, and maybe some cannot be ordered, but adding order does not make them any less categorical.

As Wikipedia says, categorical distributions are generalizations of the Bernoulli distribution to more than two possible values (the Bernoulli distribution is strictly binary). The Bernoulli distribution is also a special case of the binomial distribution, but I wouldn't call the binomial distribution categorical (it's discrete, but a count variable, so distances between values are defined). Multinomial distributions may be conflated with categorical distributions, but Wikipedia cautions against this.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.