Questions tagged [dirichlet-distribution]

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

Filter by
Sorted by
Tagged with
1
vote
1answer
32 views

Upgrading weight parameters to random variable in Gaussian mixtures

In a Gaussian mixture model we model a density like: $p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1] where $\pi,\mu$ and $\sigma$ are parameters. I would like to know if the ...
3
votes
1answer
97 views

Difficulties in computing the derivatives of the Dirichlet distribution

I need to compute the first derivatives of the Dirichlet distribution, defined in the following way: $$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\...
0
votes
0answers
13 views

Calculate the Wasserstein distance between the dirichlet and uniform distributions

I am reading a couple of papers that each have their own way of constructing skewed data partitions from a uniform (global) data set. The data set consists of entries with a label $y$, with in total $...
2
votes
0answers
8 views

Question about the distribution of the average of Dirichlet-distributed random variables

Suppose that each in a set of $n$ random variables $\boldsymbol{X}_1, .., \boldsymbol{X}_n$ are Dirichlet-distributed with parameters $\boldsymbol{\alpha}_i$, where $i$ is an index for the random ...
2
votes
0answers
18 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
1
vote
0answers
10 views

On the distribution of a scaled sum of a Dirchlet random variable

Consider $(X_{1},\dots,X_{K})=X\sim \text{Dir}(\alpha)$ and a vector $v=(v_{1},\dots ,v_{K})\in\mathbb{R}^{K}$. Is there a parametric density function for the distribution of: $Xv^{T}=vX^T=\sum^{K}_{i=...
0
votes
0answers
13 views

I'm trying to identify the posterior distributions in LDiA. Are theta and phi in this PGM the posterior distributions of LDiA?

My understanding is that alpha and beta are the dirichlet priors, and the posteriors would be theta and phi. And since theta and phi are from the same family of distributions as alpha and beta, the ...
0
votes
0answers
19 views

In the dirichlet distribution, does each x represent a particular distribution on the simplex?

The dirichlet distribution takes this form: I'm trying to understand how we end up with a vector from this formula rather than a scalar. The only way I see how is that each x itself is a vector ...
0
votes
0answers
21 views

Understanding Dirichlet Distribution Variance

I need some help in understanding the variance/standard deviation in the Dirichlet distribution. I apologize in advance for the lack of latex. In the Beta distribution, as the shape parameters ...
1
vote
1answer
21 views

In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?

This is the formula in question: Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
3
votes
0answers
18 views

Bounding values of a Dirichlet distribution

Consider $k$ random variables $X_1, X_2, \ldots, X_k$ such that $(X_1, X_2, \ldots, X_k)$ follow a $\text{Dirichlet}(1, 1, \ldots, 1)$ distribution. For a large enough $k$, I am trying to bound/find ...
0
votes
0answers
9 views

Bayesian network: graph synthesis & data sampling

Input (What I have): some Bayesian networks (both graph structure and conditional probability distribution (cpd)) and corresponding categorical datasets (e.g. bnlearn repo). Output (What I want): ...
3
votes
0answers
90 views

A clarification in the original Dirichlet Process paper by Ferguson

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
0
votes
0answers
20 views

Estimate parameters of a concrete categorical mixture model (information retrieval)

Let $f_{i,d}$ be the frequency of the word $i$ in the document $d$ and $l_d$ be the length of the document $d$. Then $P(X = i \mid D = d) = \frac{f_{i,d}}{l_{d}}$ is the probability of drawing the ...
0
votes
0answers
11 views

Distribution of the mean of a Dirichlet-distributed distribution

Suppose that $(f_0,\dotsc,f_N)$, with $f_n\ge0, \sum_n f_n=1$, is a distribution (set of normalized weights or frequencies) having a Dirichlet distribution with parameters $\alpha_n$: $$\mathrm{p}(f_0,...
0
votes
0answers
12 views

How to parameterize variational Dirichlet distribution

I am learning about variational inference and am implementing a couple of things from scratch. I am trying to build a Gaussian mixture model where the prior on the mixture component selection is a ...
4
votes
1answer
130 views

Deriving the marginal multivariate Dirichlet distribution

I am trying to understand how my professor (see derivation below) has derived the multivariate marginal distribution of a subvector of $\theta_j$´s from a Dirichlet distribution. I understand ...
1
vote
1answer
36 views

Parametrization of Dirichlet distribution

Hej! Consider I have a Dirichlet distribution with 4 variables, where the mean (u) values of these are known. $(u1+u2+u3+u4=1)$ Now, I want to obtain the parameters of the Dirichlet distribution ($\...
3
votes
0answers
138 views

Using categorical data to build a Dirichlet distribution

I am building a graphical model. I have some categorical data $\boldsymbol{\mu}$ where they are generated by $p(\boldsymbol{\mu}|\boldsymbol{s},\mathbf{A})=\prod_k\prod_j\mathbf{A}_{ij}^{\mu_is_j}$. ...
0
votes
0answers
21 views

Cumulative sum of a Dirichlet distribution?

Assume $X_1, X_2, ..., X_n \sim Dir(\alpha_1, \alpha_2, ..., \alpha_n)$ What is the joint distribution of $X_1, X_1+X_2, ..., \sum_{i=1}^n X_i$?
0
votes
1answer
59 views

Calculate log likelihood of Dirichlet distribution using Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...
0
votes
2answers
37 views

What Distribution Do I need?

Suppose I am drawing coloured balls from a bag. The ball can be red, green or blue. The probabilities of drawing a red, green or blue bag are uncertain, but I have confidence bounds for the ...
0
votes
0answers
11 views

Dirichlet process Posterior notation

While reading through Yee Whye Teh's tutorial on Dirichlet Process, I came across the Posterior distribution for DP. After observing some draws $\theta_1,\theta_2, ... ,$, the posterior is defined as $...
0
votes
0answers
13 views

Measure of base distribution $G_0(A)$ in the Dirichlet process

I am studying the Dirichlet Process and I came across this rule that states that given a partition $A_i's$ over the space $\Theta$, which represents the space in which $G_0$ is drawn from, $$(G(A_1),G(...
5
votes
2answers
345 views

Is there a statistical distribution whose values are bounded $[-1,1]$ and sum to 1?

The Dirichlet distribution contains values that are bounded $[0,1]\in \mathbb{R}$ and sum to $1$. Is there a parametric distribution or similar method whose values do the same but reach as low as $-1$?...
2
votes
1answer
79 views

Probability that a Linear Combination of Dirichlet Random Variables is a Distribution

I've been putting a lot of thought on this problem, but it seems I ran out of ideas. Any help would be appreciated! Suppose we generate two probability vectors $\boldsymbol{\theta}_1, \boldsymbol{\...
2
votes
0answers
14 views

Restriction and dependence in the Dirichlet distributon

The Dirichlet distribution is sometimes said that it is "too restrictive and imposes strong conditions on the dependence between components". What is the reason?
0
votes
0answers
40 views

Overall variance of a Dirichlet distribution

I have observations on proportion of individuals at different age groups. I'm doing some simulation experiments, and I need to introduce error in the observed data. For that, I'm simulating data from ...
1
vote
0answers
16 views

Marginal Cluster assignments for Dirichlet Process mixture model

I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
0
votes
1answer
142 views

How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution

I have derived the mean and variance of $\mu_j$ in Dirichlet distribution $\text{Dir}(\mu_1, \cdots, \mu_K|\alpha_1, \cdots, \alpha_k)$. On https://en.wikipedia.org/wiki/Dirichlet_distribution, it ...
3
votes
2answers
221 views

What is a non-informative choice of parameters for a Dirichlet distribution?

Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
0
votes
0answers
48 views

On the choice of prior in Bayesian Bootstrap

Let $d=(d_1,…,d_K)$ be a vector of all the possible values that the data $x=(x_1,…,x_N)$ could possibly take. Then, each $x_i$ is modeled as being drawn from the $K$ possible values where the ...
0
votes
1answer
137 views

Uniform posterior on bounded space [duplicate]

In a particular Bayesian problem, I have encountered a choice of parameters that leads to a uniform posterior distribution. Given prior \begin{equation} p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\...
1
vote
0answers
23 views

How to model proportions with a hierarchical structure?

I have thinking about how to model proportions for a problem with hierarchical structure. In the problem, I have observations of users over multiple days, where each observation is a proportion of ...
0
votes
2answers
233 views

Softmax vs the Dirichlet distribution

As far as I understand one can in principle model the distribution over a set of $k$ categories using e.g.: the Dirichlet distribution A softmax model. As far as I can tell, both use $k$ parameters ...
0
votes
1answer
64 views

How does the uniform Dirichlet PDF integrate to 1?

For the uninformative 3-dimensional Dirichlet prior ${\rm Dir}(1, 1, 1)$, I understand that the probability density function (PDF) evaluates uniformly to 2, and the support are all three-dimensional ...
2
votes
2answers
261 views

On posterior in the Bayesian bootstrap

The Bayesian bootstrap was introduced by Rubin (1981) as a Bayesian analog of the original bootstrap. Given dataset $X=\{x_1, \dots, x_N\}$, instead of drawing weights $\pi_{n}$ from the discrete set $...
0
votes
0answers
15 views

Finding the log Jacobian of a transformation from lower to high dimensions, specifically in the case of normal cdf to Dirichlet?

I have random variables $\mu$ and $\sigma$ that I have transformed and I am interested in finding their joint distribution given the following information I have. Particularly, I need help finding the ...
0
votes
2answers
64 views

Fitting/Inference for Dirichlet Process/CRP for clustering

Please excuse my ignorance on this topic, I don't have much experience in nonparametric Bayes. I read about Dirichlet process clustering and the Chinese Restaurant Process analogies. I think I ...
0
votes
0answers
12 views

Understanding Dirichlet clustering

I am trying to understand Dirichlet processes and how they are used for clustering. I am following this overview. I understand Dirichlet distributions (ie. generalizations of the Beta, used for ...
3
votes
2answers
667 views

Dirichlet distribution vs Multinomial distribution?

Both Dirichlet and multinomial distributions are distributions over vectors, and both Dirichlet and multinomial distributions are constrained so that all of the elements of these vectors sum to a ...
0
votes
0answers
13 views

log likelihood function causes huge differences with similar inputs

I am not sure whether there is a solution to this problem, but here goes. My problem is that in my function, I am taking log of a matrix and then taking its mean, and doing this for two very similar ...
0
votes
0answers
44 views

Posterior probability distribution from multinomial sample

I want to get the posterior from a multinomial sample and want to know if the following derivation is correct. Suppose when drawing (with replacement) a sample of $N$ balls from a urn with $K$ ...
1
vote
1answer
250 views

Why use MCMC sampling when using conjugate priors?

I've been getting to grips with some Bayesian modelling, but one thing is confusing the heck out of me when I look at tutorials and worked-through problems online. I'm looking at a problem with a ...
0
votes
0answers
22 views

Question about possible typo in a tutorial about the stick-breaking model of the Dirichlet distribution

I am reading a tutorial on the Dirichlet distribution: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf and I think there is a typo in Step 2 of the stick-breaking model of ...
2
votes
0answers
27 views

Proving independence relationship

Let $X_1,X_2,X_3$ be continuous positive random variables satisfying $X_1+X_2+X_3<1$ and the following independence relations $$\frac{X_1}{X_1+X_2}\perp \!\!\!\perp \frac{X_3}{1-X_1-X_2}~ and$$ $$\...
1
vote
0answers
146 views

Proposal Function For Variables That Sum To 1 (Dirichlet Prior)

Recently I've been trying to use MCMC to infer a set of 50 random variables (species frequencies) that sum to 1 with the Metropolis-Hastings algorithm. However, the algorithm is not working well ...
0
votes
1answer
45 views

Normalized subvectors of Dirichlet, mutually independent?

Let$$X=(X_1,\cdots,X_k)\sim Dir(\alpha_1,\cdots,\alpha_k)$$ According to this reference, the independence of the two vectors, $$\bigg(\frac{X_1}{X_1+\cdots+X_j},\cdots,\frac{X_j}{X_1+\cdots+X_j}\...
1
vote
1answer
103 views

Inference on Dirichlet hyper-parameter

I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model. In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$...
0
votes
1answer
65 views

LDA alpha equivalent in structural topic model

I'm using an implementation of the structural topic model (stm), written in R using the stm package. I want to reduce the number of topics that are prevalent in ...

1
2 3 4 5 6