114
votes
Accepted
What exactly is the alpha in the Dirichlet distribution?
The Dirichlet distribution is a multivariate probability distribution that describes $k\ge2$ variables $X_1,\dots,X_k$, such that each $x_i \in (0,1)$ and $\sum_{i=1}^N x_i = 1$, that is parametrized ...
- 141k
24
votes
What exactly is the alpha in the Dirichlet distribution?
Disclaimer: I have never worked with this distribution before. This answer is based on this wikipedia article and my interpretation of it.
The Dirichlet distribution is a multivariate probability ...
- 1,043
18
votes
Purpose of Dirichlet noise in the AlphaZero paper
Question 1 is straightforward, here $\alpha$ is a vector of repetitions of the given value. (As answered by Max S.)
Question 2 is more interesting: The Dirichlet distribution has the following ...
- 281
10
votes
The meaning of representing the simplex as a triangle surface in Dirichlet distribution?
I don't understand what is the role of the triangle here. What is it trying to communicate or visualize?
All points in the triangle must satisfy the two constraints: between zero and one in each ...
- 368
10
votes
Accepted
What happens when merging random variables in Dirichlet distribution?
It is a Dirichlet distribution having the expected parameters.
To see this, note that the vector-valued random variable $\mathbf{X}=(X_1, X_2, \ldots, X_k)$ has the same distribution as the variable
...
- 333k
10
votes
Accepted
Dirichlet distribution vs Multinomial distribution?
Multinomial distribution is a discrete, multivariate distribution for $k$ variables $x_1,x_2,\dots,x_k$ where each $x_i \in \{0,1,\dots,n\}$ and $\sum_{i=1}^k x_i = n$. Dirichlet distribution is a ...
- 141k
10
votes
Is there a statistical distribution whose values are bounded $[-1,1]$ and sum to 1?
Scaling a Dirichlet distribution
If you want a variable that is distributed like a Dirichlet distributed variable but with a different range then you can scale and shift (transform the variable). This ...
- 86.4k
9
votes
Accepted
Maximum of a probability vector distributed as a Dirichlet variate
I am not sure there is a closed-form solution for the distribution of
$p_{(k)}$ when
$(p_1,\ldots,p_k)\sim\text{Dir}(\alpha_1,\ldots,\alpha_k)$ and the
$\alpha_i$'s are different. At least, it ...
- 108k
8
votes
Accepted
Find marginal distribution of $K$-variate Dirichlet
The marginal distribution of $x_j$ is,
$$
p(x_j) = \frac{1}{B({\bf a})} \int_0^{1 - x_j} \int_0^{1 - x_j - x_1} \cdots \int_0^{1 - \sum_{k =1}^{K-2} x_k} \prod_{p=1}^{K-1} x_p^{a_p - 1} \left( 1 - \...
- 3,454
7
votes
Accepted
Generate a random set of numbers with fixed sum and desired means and variances?
Multivariate logit-normal distribution can be considered as a generalization of the Dirichlet distribution that you have in mind. It is parametrized by a vector of $D-1$ means $\boldsymbol{\mu}$ for $...
- 141k
7
votes
Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters
To demonstrate a solution to this hyperprior problem, I implemented an hierarchical gamma-Dirichlet-multinomial model in PyMC3. The gamma prior for the Dirichlet is specified and sampled per Ted ...
- 71
7
votes
What is a Dirichlet prior
Let me try to respond your very last question about understanding the Dirichlet distribution, its relation to the Multinomial, and what I suspect is what you really would like to know is how this ...
- 429
7
votes
Purpose of Dirichlet noise in the AlphaZero paper
For question number 1 the answer is yes, $\alpha$ is a vector, but in this case all values are the same. According to wikipedia this is called a symmetric Dirichlet distribution, and is used when "...
- 1,766
7
votes
Accepted
Why use MCMC sampling when using conjugate priors?
You are correct that if you have a conjugate prior, there's no need to use MCMC as the posterior has a closed form solution. MCMC tutorials that present a problem where we know the posterior already ...
- 21.6k
7
votes
Accepted
Deriving the marginal multivariate Dirichlet distribution
Since$$p(\theta_1,\ldots,\theta_{k-1})\propto (1-\theta_1-\cdots-\theta_{k-1})^{\alpha_{k}-1}\prod_{i=1}^{k-1} \theta_i^{\alpha_i-1}$$over the $\mathbb R^k$-simplex,
$$\mathfrak S = \left\{(\theta_1,\...
- 108k
7
votes
How to visualize Dirichlet distribution (with more than 3 targets)?
A Dirichlet distribution is a distribution over a simplex.
A simplex is a lower-dimensional ($n-1)$ subspace of an $n$-dimensional space. With two targets, the simplex is a line. With three targets, ...
- 9,001
7
votes
Accepted
CDF of Dirichlet Distribution
The Dirichlet distribution is either defined on the simplex of $\mathbb R^k$,
$$\mathcal S_{k-1}=\big\{\mathbf x;\ x_i\in (0,1),~i=1,2,\ldots,k,~\sum_{i=1}^k x_i=1\big\}$$
in which case the density
$$...
- 108k
7
votes
Accepted
Dirichlet distribution with correlated components?
My question is how can correlated components be included in a Dirichlet distribution, as in the case described above?
One approach is to assume that the elementwise log of the Dirichlet parameter ...
- 9,660
6
votes
Deriving the MAP estimate for Multinomial-Dirichlet
You can impose the constraint $\sum \theta_i = 1$ by specifying $\theta_k = 1 - \sum_{i<k}\theta_i$ in your likelihood function. This results in:
$$l(\theta) = \sum_{i=1}^{k-1}(x_i+a_i-1)\log \...
- 41k
6
votes
Accepted
How do we incorporate new information into a Dirichlet prior distribution?
Dirichlet prior is an appropriate prior, and is the conjugate prior to a multinomial distribution. However, it seems a bit tricky to apply this to the output of a multinomial logistic regression, ...
- 4,817
6
votes
Accepted
Predictive Density for Dirichlet Multinomial
The first slide applies the general result of the second slide to a case when $$y=\overbrace{(0,\ldots,0,1,0,\ldots,0)}^{1\text{ at }\kappa\text{th position}}$$ with different notations, since [in the ...
- 108k
6
votes
Accepted
How to use the Dirichlet prior for estimating the multinomial parameters?
Say that you have an urn with red, green, and blue balls, you draw $n$ balls from the urn with replacement. The distribution of the counts of the red, green, and blue balls, $(x_1, x_2, x_3)$, would ...
- 141k
5
votes
Drawing from Dirichlet distribution
A simple method (while not exact) consists in using the fact that drawing a Dirichlet distribution is equivalent to the Polya's urn experiment. (Drawing from a set of colored balls and each time you ...
- 566
5
votes
Accepted
Truncated Dirichlet process vs Dirichlet distribution
You have two very different priors on $\pi$, so why would you expect the same posterior?
Let's think about the nature of $T$ in your presentation above:
In 1: Dirichlet Distribution, $T$ is a ...
- 5,447
5
votes
Accepted
Deriving the MAP estimate for Multinomial-Dirichlet
I see two mistakes in your steps.
First of all, taking the derivative with respect to a single $\theta_m$ cancels out all the other terms where $i \ne m$ in the sum. That is,
$$
\frac{\partial}{\...
- 3,454
5
votes
Maximum likelihood estimation of a Dirichlet distribution multivariate parameters
You seem to be confusing many things in your question.
Is there any quick solution (either by any statistical software or
manual workout) to find the maximum likelihood estimates of alpha of
...
- 141k
5
votes
Accepted
Understanding Dirichlet Process Mixtures
Probably it is better to describe how one would generate data from a Dirichlet process mixture. Each line is understood to be conditional on all lines above it.
Sample $P \sim \operatorname{DP}(\...
- 9,102
5
votes
Accepted
From beta distribution to Dirichlet: Estimation of the concentrantion parameters
Beta distribution has $(0, 1)$ support, same as each of the variables jointly distributed as Ditichlet. Given $X_i \sim \mathsf{Beta}(a_i, b_i)$, if you wanted to have something like $(X_1, X_2, \dots,...
- 141k
5
votes
Accepted
Uniform posterior on bounded space
NOTE: My answer was written based on an earlier formulation of the question, namely on how having a uniform Dirichlet $\mathcal D(1,\ldots,1)$ posterior was at all possible. I proposed this setting as ...
- 108k
5
votes
Accepted
How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution
\begin{align}
\mathbb{E}[\ln \mu_j]
&= \int_0^1 \ln \mu_j \text{Dir}(\boldsymbol{\mu}|\boldsymbol{\alpha}) d\mu_j \\
&= \int_0^1 \ln \mu_j \text{Beta}(\alpha_j, \alpha_0 - \alpha_j) d\mu_j \\
&...
- 1,176
Only top scored, non community-wiki answers of a minimum length are eligible