# What does it mean to sample a probability vector from a Dirichlet distribution?

I'm essentially learning about Latent Dirichlet Allocation. I'm watching a video here: http://videolectures.net/mlss09uk_blei_tm/ and stuck at minute 45 when he started to explain on sampling from the distribution.

Also I tried to consult a machine learning book that doesn't have a detailed introductory on Dirichelt distribution. In the book I'm reading it mentioned an example on sampling "probability vectors" from the Dirichlet distribution, but what does that mean?

I understand sampling from a distribution as getting random values for the random variables according to the distribution. So let p_X,Y(x,y) but the pmf of any distribution, sampling from this distribuiton means I get a random (x,y) (i.e. random values for x and y). To get the probablity of the getting the event (X=x AND Y=y) we evalute the pmf of the distribution ... so we get only one number. But what are "probability vectors" here!!

I attached a screenshot for the book. I really hope you can help!

• I guess the probability vector is just what you sample with the Dirichlet distribution. Example : (0.5, 0.4, 0.1) is a vector and is used to represent the proportions/probabilities of a distribution of a variable with 3 classes. Jan 3 '14 at 13:50
• @Scratch when you said 3 classes, that means a random variable defined only on 3 discrete values, right? Jan 3 '14 at 14:16
• So basically each sample from a dirichlet represents a distribution over K classes. Jan 3 '14 at 14:22
• yes the Dirichlet distribution was created for these type of problems : simulating a distribution over classes. Jan 3 '14 at 14:31
• @Scratch can you please see my question here stats.stackexchange.com/questions/81136/… Jan 3 '14 at 15:32

If you're familiar with the beta distribution, the Dirichlet distribution might become even more clear. A beta distribution is often used to describe a distribution of probabilities of dichotomous events, so its restricted to the unit interval. For example, for a Bernoulli trial, there is only a parameter $\theta$ describing the probability of a "success." Often we think of $\theta$ as being fixed, but if we are uncertain about the "true" value of $\theta$, we could think about a distribution of all possible $\theta$s, with a larger likelihood for those we consider more plausible, so perhaps $\theta \sim \text{B}(\alpha, \beta)$, where $\alpha>\beta$ concentrates more of the mass near 1 and $\beta > \alpha$ concentrates more of the mass near 0.
One might object that the beta distribution only describes the probability of a single probability, that is, for example, that $P(\theta<0.25)=0.5$, which is a scalar number. But keep in mind that the beta distribution is describing dichotomous outcomes. So by applying Kolmogorov's second axiom, we also know that $P(\theta \ge 0.25)=0.5$ as well. Collecting these results in a vector gives us a vector of probabilities.