What is meant by categorical distribution? Is it a separate type of distribution (EX: Binomial,bernoulli, Multinomial) or any distribution that can be represented this way? Can someone elaborate with a simple example
 A: Categorical variables have finite sets of discrete values. Examples include sex (male/female), country, planet, etc. Contrast this with continuous variables, which can take an infinite number of different values. Examples include weight, longitude, distance, etc.
Note that similar information can sometimes be expressed in categorical and continuous ways; e.g., planet = earth could be expressed as distance to sun = 1 astronomical unit ≈ 150 million kilometers. However, there's not really any way to express 200 million kilometers from the sun in terms of planets, because there's no planet there (Mars is 228 million km from the sun). Same for 201 million km, 202, etc. All you could say about these distances in terms of planets is planet = none; you couldn't say planet = 4/3×earth or .88×Mars, because there's no meaningful way to multiply a planet or any other categorical variable. In terms of planets, these distances would be indistinguishable, but of course they make sense as distinct distances from the sun when expressed as such – as a continuous variable.
One can also express continuous variables with arbitrary precision (e.g., one astronomical unit is 149,597,871 km, not exactly 150 million km). Conversely, there is no way to express planet = earth more precisely; Earth is exactly earth, no more nor less. Furthermore, it would not make sense to say any other planet is "more" or "less" than Earth if planet is a nominal variable. It could be coded as an ordered (ordinal) variable though – planets are ordered in terms of distance to the sun, volume, number of moons, etc. These numbers are all continuous in their own terms (or at least counts, which are discrete but not categorical), but not in terms of planets. E.g., if planets are ordered by distance from the sun or by number of moons, mars > earth > venus. If planets are ordered by volume, earth > venus > mars. It is not necessary to order categorical variables, and maybe some cannot be ordered, but adding order does not make them any less categorical.
As Wikipedia says, categorical distributions are generalizations of the Bernoulli distribution to more than two possible values (the Bernoulli distribution is strictly binary). The Bernoulli distribution is also a special case of the binomial distribution, but I wouldn't call the binomial distribution categorical (it's discrete, but a count variable, so distances between values are defined). Multinomial distributions may be conflated with categorical distributions, but Wikipedia cautions against this.
A: The categorical distribution is the generalization of the Bernoulli distribution to a fixed number $2 \le k$ of outcomes.
Equivalently, it is the special case of the multinomial distribution where the number of "choices" $n$ is fixed at one.
Therefore, it has pdf:
$$\prod_{i=1}^k p_i^{x_i}
\qquad\text{(where
$0\le p_i$
and
$\sum_i p_i = 1$)}$$
over the support
$$x_i \in \{0,1\}$$
where
$$n \triangleq \sum_{i=1}^k x_i = 1.$$
In summary, Bernoulli has $k=2, n=1$, binomial has $k=2, n\ge 1$, multinomial has $k\ge2, n\ge1$, and categorical has $k\ge2, n=1$.
