Questions tagged [dirichlet-distribution]

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

Filter by
Sorted by
Tagged with
3 votes
1 answer
333 views

Difficulties in computing the derivatives of the Dirichlet distribution

I need to compute the first derivatives of the Dirichlet distribution, defined in the following way: $$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\...
Bibi's user avatar
  • 57
2 votes
0 answers
84 views

Question about the distribution of the average of Dirichlet-distributed random variables

Suppose that each in a set of $n$ random variables $\boldsymbol{X}_1, .., \boldsymbol{X}_n$ are Dirichlet-distributed with parameters $\boldsymbol{\alpha}_i$, where $i$ is an index for the random ...
Brash Equilibrium's user avatar
2 votes
0 answers
159 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
econ86's user avatar
  • 387
1 vote
0 answers
60 views

On the distribution of a scaled sum of a Dirchlet random variable

Consider $(X_{1},\dots,X_{K})=X\sim \text{Dir}(\alpha)$ and a vector $v=(v_{1},\dots ,v_{K})\in\mathbb{R}^{K}$. Is there a parametric density function for the distribution of: $Xv^{T}=vX^T=\sum^{K}_{i=...
MHDZAHD93's user avatar
1 vote
1 answer
48 views

In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?

This is the formula in question: Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
Bob Odenkirk's user avatar
3 votes
0 answers
70 views

Bounding values of a Dirichlet distribution

Consider $k$ random variables $X_1, X_2, \ldots, X_k$ such that $(X_1, X_2, \ldots, X_k)$ follow a $\text{Dirichlet}(1, 1, \ldots, 1)$ distribution. For a large enough $k$, I am trying to bound/find ...
BlackHat18's user avatar
3 votes
0 answers
129 views

A clarification in the original Dirichlet Process paper by Ferguson

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
honeybadger's user avatar
  • 1,562
1 vote
0 answers
102 views

Distribution of the mean of a Dirichlet-distributed distribution

Suppose that $(f_0,\dotsc,f_N)$, with $f_n\ge0, \sum_n f_n=1$, is a distribution (set of normalized weights or frequencies) having a Dirichlet distribution with parameters $\alpha_n$: $$\mathrm{p}(f_0,...
pglpm's user avatar
  • 1,296
5 votes
1 answer
634 views

Deriving the marginal multivariate Dirichlet distribution

I am trying to understand how my professor (see derivation below) has derived the multivariate marginal distribution of a subvector of $\theta_j$´s from a Dirichlet distribution. I understand ...
xxtensionxx's user avatar
1 vote
1 answer
342 views

Parametrization of Dirichlet distribution

Hej! Consider I have a Dirichlet distribution with 4 variables, where the mean (u) values of these are known. $(u1+u2+u3+u4=1)$ Now, I want to obtain the parameters of the Dirichlet distribution ($\...
AAndersson's user avatar
1 vote
1 answer
251 views

Calculate log likelihood of Dirichlet distribution using Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...
alryosha's user avatar
  • 251
0 votes
2 answers
40 views

What Distribution Do I need?

Suppose I am drawing coloured balls from a bag. The ball can be red, green or blue. The probabilities of drawing a red, green or blue bag are uncertain, but I have confidence bounds for the ...
user847663's user avatar
5 votes
3 answers
1k views

Is there a statistical distribution whose values are bounded $[-1,1]$ and sum to 1?

The Dirichlet distribution contains values that are bounded $[0,1]\in \mathbb{R}$ and sum to $1$. Is there a parametric distribution or similar method whose values do the same but reach as low as $-1$?...
develarist's user avatar
  • 3,887
2 votes
1 answer
405 views

Probability that a Linear Combination of Dirichlet Random Variables is a Distribution

I've been putting a lot of thought on this problem, but it seems I ran out of ideas. Any help would be appreciated! Suppose we generate two probability vectors $\boldsymbol{\theta}_1, \boldsymbol{\...
JFarias's user avatar
  • 80
2 votes
0 answers
35 views

Restriction and dependence in the Dirichlet distributon

The Dirichlet distribution is sometimes said that it is "too restrictive and imposes strong conditions on the dependence between components". What is the reason?
Tky's user avatar
  • 139
0 votes
0 answers
282 views

Overall variance of a Dirichlet distribution

I have observations on proportion of individuals at different age groups. I'm doing some simulation experiments, and I need to introduce error in the observed data. For that, I'm simulating data from ...
Val's user avatar
  • 1
1 vote
0 answers
46 views

Marginal Cluster assignments for Dirichlet Process mixture model

I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
calveeen's user avatar
  • 1,126
3 votes
2 answers
2k views

How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution

I have derived the mean and variance of $\mu_j$ in Dirichlet distribution $\text{Dir}(\mu_1, \cdots, \mu_K|\alpha_1, \cdots, \alpha_k)$. On https://en.wikipedia.org/wiki/Dirichlet_distribution, it ...
zyxue's user avatar
  • 1,125
3 votes
2 answers
2k views

What is a non-informative choice of parameters for a Dirichlet distribution?

Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
Blade's user avatar
  • 655
0 votes
1 answer
186 views

Uniform posterior on bounded space [duplicate]

In a particular Bayesian problem, I have encountered a choice of parameters that leads to a uniform posterior distribution. Given prior \begin{equation} p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\...
Blade's user avatar
  • 655
1 vote
0 answers
140 views

How to model proportions with a hierarchical structure?

I have thinking about how to model proportions for a problem with hierarchical structure. In the problem, I have observations of users over multiple days, where each observation is a proportion of ...
Jeff's user avatar
  • 151
4 votes
3 answers
2k views

Softmax vs the Dirichlet distribution

As far as I understand one can in principle model the distribution over a set of $k$ categories using e.g.: the Dirichlet distribution A softmax model. As far as I can tell, both use $k$ parameters ...
Josh's user avatar
  • 4,448
1 vote
1 answer
194 views

How does the uniform Dirichlet PDF integrate to 1?

For the uninformative 3-dimensional Dirichlet prior ${\rm Dir}(1, 1, 1)$, I understand that the probability density function (PDF) evaluates uniformly to 2, and the support are all three-dimensional ...
Justin Wang's user avatar
3 votes
2 answers
594 views

On posterior in the Bayesian bootstrap

The Bayesian bootstrap was introduced by Rubin (1981) as a Bayesian analog of the original bootstrap. Given dataset $X=\{x_1, \dots, x_N\}$, instead of drawing weights $\pi_{n}$ from the discrete set $...
Blade's user avatar
  • 655
0 votes
2 answers
336 views

Fitting/Inference for Dirichlet Process/CRP for clustering

Please excuse my ignorance on this topic, I don't have much experience in nonparametric Bayes. I read about Dirichlet process clustering and the Chinese Restaurant Process analogies. I think I ...
bayesianpower's user avatar
6 votes
2 answers
4k views

Dirichlet distribution vs Multinomial distribution?

Both Dirichlet and multinomial distributions are distributions over vectors, and both Dirichlet and multinomial distributions are constrained so that all of the elements of these vectors sum to a ...
Thalassophile's user avatar
3 votes
1 answer
1k views

Why use MCMC sampling when using conjugate priors?

I've been getting to grips with some Bayesian modelling, but one thing is confusing the heck out of me when I look at tutorials and worked-through problems online. I'm looking at a problem with a ...
HereItIs's user avatar
  • 121
0 votes
0 answers
25 views

Question about possible typo in a tutorial about the stick-breaking model of the Dirichlet distribution

I am reading a tutorial on the Dirichlet distribution: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf and I think there is a typo in Step 2 of the stick-breaking model of ...
Noppawee Apichonpongpan's user avatar
2 votes
0 answers
34 views

Proving independence relationship

Let $X_1,X_2,X_3$ be continuous positive random variables satisfying $X_1+X_2+X_3<1$ and the following independence relations $$\frac{X_1}{X_1+X_2}\perp \!\!\!\perp \frac{X_3}{1-X_1-X_2}~ and$$ $$\...
Andeanlll's user avatar
  • 413
1 vote
0 answers
524 views

Proposal Function For Variables That Sum To 1 (Dirichlet Prior)

Recently I've been trying to use MCMC to infer a set of 50 random variables (species frequencies) that sum to 1 with the Metropolis-Hastings algorithm. However, the algorithm is not working well ...
Correlations's user avatar
0 votes
1 answer
89 views

Normalized subvectors of Dirichlet, mutually independent?

Let$$X=(X_1,\cdots,X_k)\sim Dir(\alpha_1,\cdots,\alpha_k)$$ According to this reference, the independence of the two vectors, $$\bigg(\frac{X_1}{X_1+\cdots+X_j},\cdots,\frac{X_j}{X_1+\cdots+X_j}\...
Andeanlll's user avatar
  • 413
1 vote
1 answer
480 views

Inference on Dirichlet hyper-parameter

I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model. In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$...
yassem's user avatar
  • 153
0 votes
1 answer
151 views

LDA alpha equivalent in structural topic model

I'm using an implementation of the structural topic model (stm), written in R using the stm package. I want to reduce the number of topics that are prevalent in ...
James's user avatar
  • 25
0 votes
1 answer
115 views

If (x1,x2,...xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it?

If (x1,x2,...xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it? Note that xi are all positive and summation(xi)=1
Murali's user avatar
  • 143
-1 votes
1 answer
152 views

I want to represent x1, x2, ..., xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,... have the same pdf

I want to represent x1, x2, ..., xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,...,xn have the same probability density function? all 0 < xi < 1. In ...
Murali's user avatar
  • 143
2 votes
1 answer
808 views

could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?

I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution. To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
JJJohn's user avatar
  • 1,875
6 votes
1 answer
619 views

Aggregation with an overlap: Dirichlet distribution

Suppose that we have $$(p_1,p_2,p_3,p_4)\sim Dirichlet(a_1,a_2,a_3,a_4),$$ where $p_4=1-p_1-p_2-p_3.$ When we add random variables for example, $p_1+p_2$ and $p_3+p_4$, the resulting distributions ...
Andeanlll's user avatar
  • 413
4 votes
1 answer
79 views

What is the name of this distribution?

I came across this: a categorical distribution with $K=10,000$ parameters (categories), and we take only few samples from this distribution, say $N=400$ (the point is $N < K$). Now, obviously, not ...
John Deterious's user avatar
1 vote
2 answers
825 views

Is it possible to fit a Dirichlet Regression to changing response variables?

A toy problem to illustrate my issue: We put a 100 people in a room with 10 candy bars. Each bar is different and has a different brand, flavor, size, color, etc. We ask each person in the group to ...
George's user avatar
  • 145
2 votes
1 answer
189 views

Bayesian inference out of partial information - Dirichlet example

Suppose we have two coins $X_1$ and $X_2$. They are possibly biased and correlated coins. The heads probability of each coins is denoted by $p_1$ and $p_2$ which we don't know at the beginning. The ...
Andeanlll's user avatar
  • 413
3 votes
2 answers
365 views

Probability that a random variable is smaller than another in a random vector

Suppose that a random vector $X=(X_1,X_2,X_3)$ follows a Dirichlet distribution with a shape parameter $(a_1,a_2,a_3).$ What I want to calculate is the probability of $X_1>X_2$ and I want to ...
Greenteamaniac's user avatar
2 votes
2 answers
2k views

Bayesian update for Beta distribution

I'm wondering how to find a posterior of a beta distribution when the "new information" is not an outcome of a binomial trial. Let $p$ be the probability of Head of a (biased) coin toss. As usual in ...
Andeanlll's user avatar
  • 413
1 vote
1 answer
234 views

Bayesian Inference: Prior in Chinese Restaurant Process

For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
MJon10's user avatar
  • 13
1 vote
0 answers
103 views

Treating missing data in making Bayesian inference

Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other. In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's ...
Andeanlll's user avatar
  • 413
2 votes
0 answers
79 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
Sempai's user avatar
  • 21
0 votes
1 answer
131 views

Bayesian inference using Dirichlet: muddled outcome case

In relation to my previous question (Bayesian inference for Beta distribution after an uncertain outcome), Suppose that $$(x_1,x_2,x_3)\sim Dirichlet(a_1,a_2,a_3)$$ and an associated Mutinoulli ...
Andeanlll's user avatar
  • 413
2 votes
0 answers
159 views

Difference between two dimensions sampled from Dirichlet distribution

Say I'm doing Bayesian inference on a Dirichlet-Multinomial model: $$ x \in [1,2,3]; \\ x \sim Multinomial(p_1, p_2, p_3); \\ p_1, p_2, p_3 \sim Dirichlet(\alpha_1, \alpha_2, \alpha_3); \\ \alpha_n =...
Eoin's user avatar
  • 8,987
3 votes
1 answer
1k views

From beta distribution to Dirichlet: Estimation of the concentrantion parameters

Searching at least 3 hours about the connection between beta distribution and dirichlet. My problem is: I have a collection of random variables $X_i \sim Beta(a_i, b_i)$. The parameters $a_i$ and $...
AlexandrosB's user avatar
3 votes
1 answer
185 views

Applying de Finetti's representation theorem to Dirichlet distribution

Let's begin from the de Finetti–Hewitt–Savage theorem: for an exchangeable sequence of random variables we can always write $$ p(x_1, x_2,\cdots) = \int \prod p(x_i | L) P(dL) $$ where $L$ is a latent ...
innisfree's user avatar
  • 1,470
2 votes
1 answer
280 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
tisPrimeTime's user avatar

1
2
3 4 5
7