Questions tagged [dirichlet-distribution]
The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.
312
questions
3
votes
1
answer
333
views
Difficulties in computing the derivatives of the Dirichlet distribution
I need to compute the first derivatives of the Dirichlet distribution, defined in the following way:
$$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\...
2
votes
0
answers
84
views
Question about the distribution of the average of Dirichlet-distributed random variables
Suppose that each in a set of $n$ random variables $\boldsymbol{X}_1, .., \boldsymbol{X}_n$ are Dirichlet-distributed with parameters $\boldsymbol{\alpha}_i$, where $i$ is an index for the random ...
2
votes
0
answers
159
views
Choosing the Dirichlet prior in a mixture model
Consider the following mixture model with $K < \infty$ components,
$$
f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
1
vote
0
answers
60
views
On the distribution of a scaled sum of a Dirchlet random variable
Consider $(X_{1},\dots,X_{K})=X\sim \text{Dir}(\alpha)$ and a vector $v=(v_{1},\dots ,v_{K})\in\mathbb{R}^{K}$.
Is there a parametric density function for the distribution of:
$Xv^{T}=vX^T=\sum^{K}_{i=...
1
vote
1
answer
48
views
In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?
This is the formula in question:
Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
3
votes
0
answers
70
views
Bounding values of a Dirichlet distribution
Consider $k$ random variables $X_1, X_2, \ldots, X_k$ such that $(X_1, X_2, \ldots, X_k)$ follow a $\text{Dirichlet}(1, 1, \ldots, 1)$ distribution. For a large enough $k$, I am trying to bound/find ...
3
votes
0
answers
129
views
A clarification in the original Dirichlet Process paper by Ferguson
I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
1
vote
0
answers
102
views
Distribution of the mean of a Dirichlet-distributed distribution
Suppose that $(f_0,\dotsc,f_N)$, with $f_n\ge0, \sum_n f_n=1$, is a distribution (set of normalized weights or frequencies) having a Dirichlet distribution with parameters $\alpha_n$:
$$\mathrm{p}(f_0,...
5
votes
1
answer
634
views
Deriving the marginal multivariate Dirichlet distribution
I am trying to understand how my professor (see derivation below) has derived the multivariate marginal distribution of a subvector of $\theta_j$´s from a Dirichlet distribution. I understand ...
1
vote
1
answer
342
views
Parametrization of Dirichlet distribution
Hej!
Consider I have a Dirichlet distribution with 4 variables, where the mean (u) values of these are known. $(u1+u2+u3+u4=1)$
Now, I want to obtain the parameters of the Dirichlet distribution ($\...
1
vote
1
answer
251
views
Calculate log likelihood of Dirichlet distribution using Gamma distribution
Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...
0
votes
2
answers
40
views
What Distribution Do I need?
Suppose I am drawing coloured balls from a bag.
The ball can be red, green or blue.
The probabilities of drawing a red, green or blue bag are uncertain, but I have confidence bounds for the ...
5
votes
3
answers
1k
views
Is there a statistical distribution whose values are bounded $[-1,1]$ and sum to 1?
The Dirichlet distribution contains values that are bounded $[0,1]\in \mathbb{R}$ and sum to $1$. Is there a parametric distribution or similar method whose values do the same but reach as low as $-1$?...
2
votes
1
answer
405
views
Probability that a Linear Combination of Dirichlet Random Variables is a Distribution
I've been putting a lot of thought on this problem, but it seems I ran out of ideas. Any help would be appreciated! Suppose we generate two probability vectors $\boldsymbol{\theta}_1, \boldsymbol{\...
2
votes
0
answers
35
views
Restriction and dependence in the Dirichlet distributon
The Dirichlet distribution is sometimes said that it is "too restrictive and imposes strong conditions on the dependence between components". What is the reason?
0
votes
0
answers
282
views
Overall variance of a Dirichlet distribution
I have observations on proportion of individuals at different age groups. I'm doing some simulation experiments, and I need to introduce error in the observed data. For that, I'm simulating data from ...
1
vote
0
answers
46
views
Marginal Cluster assignments for Dirichlet Process mixture model
I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
3
votes
2
answers
2k
views
How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution
I have derived the mean and variance of $\mu_j$ in Dirichlet distribution $\text{Dir}(\mu_1, \cdots, \mu_K|\alpha_1, \cdots, \alpha_k)$.
On https://en.wikipedia.org/wiki/Dirichlet_distribution, it ...
3
votes
2
answers
2k
views
What is a non-informative choice of parameters for a Dirichlet distribution?
Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
0
votes
1
answer
186
views
Uniform posterior on bounded space [duplicate]
In a particular Bayesian problem, I have encountered a choice of parameters that leads to a uniform posterior distribution. Given prior
\begin{equation}
p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\...
1
vote
0
answers
140
views
How to model proportions with a hierarchical structure?
I have thinking about how to model proportions for a problem with hierarchical structure.
In the problem, I have observations of users over multiple days, where each observation is a proportion of ...
4
votes
3
answers
2k
views
Softmax vs the Dirichlet distribution
As far as I understand one can in principle model the distribution over a set of $k$ categories using e.g.:
the Dirichlet distribution
A softmax model.
As far as I can tell, both use $k$ parameters ...
1
vote
1
answer
194
views
How does the uniform Dirichlet PDF integrate to 1?
For the uninformative 3-dimensional Dirichlet prior ${\rm Dir}(1, 1, 1)$, I understand that the probability density function (PDF) evaluates uniformly to 2, and the support are all three-dimensional ...
3
votes
2
answers
594
views
On posterior in the Bayesian bootstrap
The Bayesian bootstrap was introduced by Rubin (1981) as a Bayesian analog of the original bootstrap. Given dataset $X=\{x_1, \dots, x_N\}$, instead of drawing weights $\pi_{n}$ from the discrete set
$...
0
votes
2
answers
336
views
Fitting/Inference for Dirichlet Process/CRP for clustering
Please excuse my ignorance on this topic, I don't have much experience in nonparametric Bayes. I read about Dirichlet process clustering and the Chinese Restaurant Process analogies. I think I ...
6
votes
2
answers
4k
views
Dirichlet distribution vs Multinomial distribution?
Both Dirichlet and multinomial distributions are distributions over vectors, and both Dirichlet and multinomial distributions are constrained so that all of the elements of these vectors sum to a ...
3
votes
1
answer
1k
views
Why use MCMC sampling when using conjugate priors?
I've been getting to grips with some Bayesian modelling, but one thing is confusing the heck out of me when I look at tutorials and worked-through problems online. I'm looking at a problem with a ...
0
votes
0
answers
25
views
Question about possible typo in a tutorial about the stick-breaking model of the Dirichlet distribution
I am reading a tutorial on the Dirichlet distribution: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf
and I think there is a typo in Step 2 of the stick-breaking model of ...
2
votes
0
answers
34
views
Proving independence relationship
Let $X_1,X_2,X_3$ be continuous positive random variables satisfying $X_1+X_2+X_3<1$ and the following independence relations
$$\frac{X_1}{X_1+X_2}\perp \!\!\!\perp \frac{X_3}{1-X_1-X_2}~ and$$
$$\...
1
vote
0
answers
524
views
Proposal Function For Variables That Sum To 1 (Dirichlet Prior)
Recently I've been trying to use MCMC to infer a set of 50 random variables (species frequencies) that sum to 1 with the Metropolis-Hastings algorithm. However, the algorithm is not working well ...
0
votes
1
answer
89
views
Normalized subvectors of Dirichlet, mutually independent?
Let$$X=(X_1,\cdots,X_k)\sim Dir(\alpha_1,\cdots,\alpha_k)$$
According to
this reference, the independence of the two vectors,
$$\bigg(\frac{X_1}{X_1+\cdots+X_j},\cdots,\frac{X_j}{X_1+\cdots+X_j}\...
1
vote
1
answer
480
views
Inference on Dirichlet hyper-parameter
I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model.
In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e.
$$...
0
votes
1
answer
151
views
LDA alpha equivalent in structural topic model
I'm using an implementation of the structural topic model (stm), written in R using the stm package.
I want to reduce the number of topics that are prevalent in ...
0
votes
1
answer
115
views
If (x1,x2,...xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it?
If (x1,x2,...xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it? Note that xi are all positive and summation(xi)=1
-1
votes
1
answer
152
views
I want to represent x1, x2, ..., xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,... have the same pdf
I want to represent x1, x2, ..., xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,...,xn have the same probability density function? all 0 < xi < 1. In ...
2
votes
1
answer
808
views
could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?
I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution.
To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
6
votes
1
answer
619
views
Aggregation with an overlap: Dirichlet distribution
Suppose that we have $$(p_1,p_2,p_3,p_4)\sim Dirichlet(a_1,a_2,a_3,a_4),$$
where $p_4=1-p_1-p_2-p_3.$
When we add random variables for example, $p_1+p_2$ and $p_3+p_4$, the resulting distributions ...
4
votes
1
answer
79
views
What is the name of this distribution?
I came across this: a categorical distribution with $K=10,000$ parameters (categories), and we take only few samples from this distribution, say $N=400$ (the point is $N < K$). Now, obviously, not ...
1
vote
2
answers
825
views
Is it possible to fit a Dirichlet Regression to changing response variables?
A toy problem to illustrate my issue:
We put a 100 people in a room with 10 candy bars. Each bar is different and has a different brand, flavor, size, color, etc. We ask each person in the group to ...
2
votes
1
answer
189
views
Bayesian inference out of partial information - Dirichlet example
Suppose we have two coins $X_1$ and $X_2$. They are possibly biased and correlated coins. The heads probability of each coins is denoted by $p_1$ and $p_2$ which we don't know at the beginning. The ...
3
votes
2
answers
365
views
Probability that a random variable is smaller than another in a random vector
Suppose that a random vector $X=(X_1,X_2,X_3)$ follows a Dirichlet distribution with a shape parameter $(a_1,a_2,a_3).$
What I want to calculate is the probability of $X_1>X_2$ and I want to ...
2
votes
2
answers
2k
views
Bayesian update for Beta distribution
I'm wondering how to find a posterior of a beta distribution when the "new information" is not an outcome of a binomial trial.
Let $p$ be the probability of Head of a (biased) coin toss. As usual in ...
1
vote
1
answer
234
views
Bayesian Inference: Prior in Chinese Restaurant Process
For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
1
vote
0
answers
103
views
Treating missing data in making Bayesian inference
Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other.
In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's ...
2
votes
0
answers
79
views
Gibbs sampling for mixture with Dirichlet prior?
I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where:
$$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
0
votes
1
answer
131
views
Bayesian inference using Dirichlet: muddled outcome case
In relation to my previous question (Bayesian inference for Beta distribution after an uncertain outcome),
Suppose that $$(x_1,x_2,x_3)\sim Dirichlet(a_1,a_2,a_3)$$
and an associated Mutinoulli ...
2
votes
0
answers
159
views
Difference between two dimensions sampled from Dirichlet distribution
Say I'm doing Bayesian inference on a Dirichlet-Multinomial model:
$$
x \in [1,2,3]; \\
x \sim Multinomial(p_1, p_2, p_3); \\
p_1, p_2, p_3 \sim Dirichlet(\alpha_1, \alpha_2, \alpha_3); \\
\alpha_n =...
3
votes
1
answer
1k
views
From beta distribution to Dirichlet: Estimation of the concentrantion parameters
Searching at least 3 hours about the connection between beta distribution and dirichlet. My problem is:
I have a collection of random variables $X_i \sim Beta(a_i, b_i)$. The parameters $a_i$ and $...
3
votes
1
answer
185
views
Applying de Finetti's representation theorem to Dirichlet distribution
Let's begin from the de Finetti–Hewitt–Savage theorem: for an exchangeable sequence of random variables we can always write
$$
p(x_1, x_2,\cdots) = \int \prod p(x_i | L) P(dL)
$$
where $L$ is a latent ...
2
votes
1
answer
280
views
Clarifying Dirichlet Process Mixture Probability Terms
Suppose I have a Dirichlet Process Mixture model defined as follows:
$\alpha \sim G(a,b)\\
\pi|\alpha \sim \text{Dir}(\alpha)\\
z|\pi \sim \text{Cat}(\pi)\\
$
where $G$ is just a standard Gamma ...