Questions tagged [dirichlet-distribution]

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

Filter by
Sorted by
Tagged with
1 vote
0 answers
26 views

Marginal density of dirichlet distribution

I'm studying BRML. In this book, a Dirichlet distribution is defined as $$ p(\alpha | u) = \frac{\Gamma(\sum_{q=1}^Q u_q)}{\prod_{q=1}^Q \Gamma(u_q)} \delta_0 \left( \sum_{q=1}^{Q} \alpha_q - 1 \right)...
  • 11
1 vote
0 answers
27 views

Comparing two randomly loaded dice

Say I have two six-sided dice, A and B, which are loaded in different ways, and I'd like to compare their probability distributions. So far I've constructed the priors for the probabilities $\vec\pi = ...
3 votes
1 answer
22 views

compute Dirichlet distribution parameter from known mean distribution

For a particular Bayesian study I am going to apply Dirichlet distribution as my proposal random number generator. I am going to update the distribution parameter every trial based on a given ...
2 votes
1 answer
33 views

Interpreting the quantities sampled from a Dirichlet distribution

Suppose you sample $M$ vectors from $Dirichlet_K(\alpha)$. You then show a histogram summarizing the distribution of the $M$ values that were sampled for dimension $k = 1$ (i.e. the first dimension, ...
0 votes
0 answers
10 views

Why does latent dirichlet allocation (LDA) fail when dealing with large and heavy-tailed vocabularies?

I'm reading the 2019 paper Topic Modeling in Embedding Spaces which claims that the embedded topic model improves on these limitations of LDA. But why does LDA have these limitations—why does it fail ...
0 votes
0 answers
21 views

Let $H$ be the base distribution of a Dirichlet process. How is this process well-defined in case $H(B_1) = 0$?

I have read that the parameters of Dirichlet distribution must be strictly positive. The Dirichlet distribution of order $K \geq 2$ with parameters $\alpha_{1}, \ldots, \alpha_{K} \color{blue}{> 0}...
  • 393
1 vote
2 answers
145 views

Bayesian updates for Dirichlet-multinomial with Gamma prior

Let $$ \begin{aligned} X_i &\sim \text{Dir-multinom}(X\mid\lambda)\\ \lambda_{j} &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)\\ \end{aligned} $$ where $i$ iterates over observations, $j$ ...
  • 213
0 votes
0 answers
19 views

Why do proportions hold across credibility range in multinomial distribution? How to compute probability of deviations from that?

I know I can use pymc3 code like this to find out if die is biased and what side is it that is more prone to show up (taken from here). ...
  • 291
0 votes
1 answer
93 views

Computing a prior from two components in Naive Bayes

Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice? More specifically, from the article of Nigam et ...
0 votes
0 answers
116 views

Variational Autoencoder with Dirichlet distributed latent space using the Weibull Distribution

My goal is to create an VAE with an Dirichlet distributed latent space. Since the reparametrization trick does not work for the Dirichlet Distribution, I am trying to approximate the Gamma ...
0 votes
0 answers
30 views

Mixtures of Dirichlet multivariates or Dirichlet processes

I am exploring the properties of Dirichlet distributions and their parameters. When mixing two Dirichlet distributed random bivariates $$\mathbf{X}\equiv(X_1,X_2)\sim\text{Dir}(\alpha_1,\alpha_2)$$ ...
1 vote
0 answers
51 views

Dirichlet distribution parameters from known variances

Let's assume, I know the variances of Dirichlet distribution parameters. Let these variances be: $Var[X_1], ..., Var[X_n]$. Is there a analytical solution to derive the parameter value alpha_i given ...
0 votes
2 answers
84 views

Sum of squares for a Dirichlet distribution

I have some data that takes the form of vectors $(a_0,...,a_n)$ lying on the simplex $\Sigma a_i = 1$ (all $a_i$'s non-negative). I have noticed that the maximum $\max_i a_i$ is very highly correlated ...
  • 3
2 votes
0 answers
230 views

Reparameterization trick for the Dirichlet distribution

Summary: My aim is to create a (probabilistic) neural network for classification that learns the distribution of its class probabilities. The Dirichlet distribution seems to be choice. I am familiar ...
0 votes
0 answers
13 views

Dirichlet distribution concentration parameter [duplicate]

Suppose I have a categorical distribution with 4 possible states and I want a Dirichlet distribution over it and I want every discrete state to be equally probable. I am slightly confused about the ...
  • 4,450
0 votes
0 answers
91 views

Using the methods of moments in R for the dirichlet distribution

I'm trying to build a distribution of transition probabilities to randomly sample from in a Markov model where individuals can transition from one health state to another (assume that in the image ...
5 votes
1 answer
477 views

CDF of Dirichlet Distribution

We know that a random variable $p=(p_{1}, p_{2},..., p_{K})$ which follows a $\textit{Dirichlet}$ distribution with parameters $\textbf{a} = (a_{1}, a_{2},..., a_{K})$ has as pdf $$f(p) = \frac{1}{B(\...
  • 1,833
0 votes
0 answers
82 views

How is a convex combination of Dirichlet-distributed variables distributed?

Let $X = (X_1, \dots, X_K) \sim \operatorname{Dir}(\alpha_1, \dots, \alpha_K)$ and define the convex combination $Y = \sum_{i=1}^{K} c_i X_i$. In the case of $K=2$, the constraint $\sum_{i=1}^{K} X_i =...
  • 1
2 votes
1 answer
57 views

Bayesian (continuous) logistic regression model with Beta likelihood?

I have a problem where my target variable are continuous/float values in the range [0,1]. If my data were integers in {0,1} this would be a simple logistic regression / Bernoulli likelihood problem. ...
  • 1,728
0 votes
0 answers
56 views

Dirichlet-distribution and its correlation?

I have the following variables that follow a beta distribution: ...
4 votes
1 answer
186 views

Mean of Generalization of the Dirichlet Distribution

I know that if $X_{1},X_{2},...X_{n}$ are independent $\mathrm{Gamma}(\alpha_{i},\theta)$ - distributed variables (notice they all have the same scale parameter $\theta$) and $Y_{i}=\frac{X_{i}}{\sum_{...
  • 53
2 votes
1 answer
288 views

How to generate data from a generalized Dirichlet distribution?

I need to generate data from a generalized Dirichlet distribution in Python to test my model, but I have no idea how can I proceed with that, can anyone guide me?
1 vote
1 answer
45 views

Concentration Bounds for categorial distribution with good Dirichlet prior

I would like to know if there are any standard methods for analyzing the concentration bounds (for example Hoeffding's bound) for a multinomial distribution modelled with a Dirichlet prior, with the ...
  • 131
4 votes
1 answer
451 views

How to visualize Dirichlet distribution (with more than 3 targets)?

I want to plot a Dirichlet distribution $\operatorname{Dir}(\alpha), \alpha=[\alpha_1, \alpha_2, \ldots,\alpha_n]$. However, when I google it, almost all of the results consider 3 targets ($n=3$), and ...
1 vote
1 answer
138 views

Stationary distribution of a Markov chain with a random transition matrix

Consider a Markov chain $\{X_t\}$ on a finite state $\mathcal{S} = \{1,\dots, S\}$ space whose transition matrix $P$ is populated by elements of the form $$ p_{ij} = P(X_{t+1} = j | X_t = i)$$ and we ...
  • 67
1 vote
0 answers
66 views

Log-likelihood of a finite mixture distribution (PDF overflowing)

I'm trying to use a finite mixture of Dirichlet distributions in a project, but am encountering problems with the PDF becoming so large for input values close to 0 that it overflows to infinity (as ...
2 votes
1 answer
64 views

Upgrading weight parameters to random variable in Gaussian mixtures

In a Gaussian mixture model we model a density like: $p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1] where $\pi,\mu$ and $\sigma$ are parameters. I would like to know if the ...
  • 703
3 votes
1 answer
199 views

Difficulties in computing the derivatives of the Dirichlet distribution

I need to compute the first derivatives of the Dirichlet distribution, defined in the following way: $$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\...
  • 57
2 votes
0 answers
37 views

Question about the distribution of the average of Dirichlet-distributed random variables

Suppose that each in a set of $n$ random variables $\boldsymbol{X}_1, .., \boldsymbol{X}_n$ are Dirichlet-distributed with parameters $\boldsymbol{\alpha}_i$, where $i$ is an index for the random ...
2 votes
0 answers
110 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
  • 388
1 vote
0 answers
27 views

On the distribution of a scaled sum of a Dirchlet random variable

Consider $(X_{1},\dots,X_{K})=X\sim \text{Dir}(\alpha)$ and a vector $v=(v_{1},\dots ,v_{K})\in\mathbb{R}^{K}$. Is there a parametric density function for the distribution of: $Xv^{T}=vX^T=\sum^{K}_{i=...
1 vote
1 answer
31 views

In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?

This is the formula in question: Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
3 votes
0 answers
36 views

Bounding values of a Dirichlet distribution

Consider $k$ random variables $X_1, X_2, \ldots, X_k$ such that $(X_1, X_2, \ldots, X_k)$ follow a $\text{Dirichlet}(1, 1, \ldots, 1)$ distribution. For a large enough $k$, I am trying to bound/find ...
3 votes
0 answers
113 views

A clarification in the original Dirichlet Process paper by Ferguson

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
  • 1,322
0 votes
0 answers
38 views

Distribution of the mean of a Dirichlet-distributed distribution

Suppose that $(f_0,\dotsc,f_N)$, with $f_n\ge0, \sum_n f_n=1$, is a distribution (set of normalized weights or frequencies) having a Dirichlet distribution with parameters $\alpha_n$: $$\mathrm{p}(f_0,...
  • 1,223
5 votes
1 answer
317 views

Deriving the marginal multivariate Dirichlet distribution

I am trying to understand how my professor (see derivation below) has derived the multivariate marginal distribution of a subvector of $\theta_j$´s from a Dirichlet distribution. I understand ...
1 vote
1 answer
116 views

Parametrization of Dirichlet distribution

Hej! Consider I have a Dirichlet distribution with 4 variables, where the mean (u) values of these are known. $(u1+u2+u3+u4=1)$ Now, I want to obtain the parameters of the Dirichlet distribution ($\...
1 vote
1 answer
161 views

Calculate log likelihood of Dirichlet distribution using Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...
  • 209
0 votes
2 answers
37 views

What Distribution Do I need?

Suppose I am drawing coloured balls from a bag. The ball can be red, green or blue. The probabilities of drawing a red, green or blue bag are uncertain, but I have confidence bounds for the ...
5 votes
2 answers
607 views

Is there a statistical distribution whose values are bounded $[-1,1]$ and sum to 1?

The Dirichlet distribution contains values that are bounded $[0,1]\in \mathbb{R}$ and sum to $1$. Is there a parametric distribution or similar method whose values do the same but reach as low as $-1$?...
  • 3,199
2 votes
1 answer
219 views

Probability that a Linear Combination of Dirichlet Random Variables is a Distribution

I've been putting a lot of thought on this problem, but it seems I ran out of ideas. Any help would be appreciated! Suppose we generate two probability vectors $\boldsymbol{\theta}_1, \boldsymbol{\...
  • 80
2 votes
0 answers
22 views

Restriction and dependence in the Dirichlet distributon

The Dirichlet distribution is sometimes said that it is "too restrictive and imposes strong conditions on the dependence between components". What is the reason?
  • 139
0 votes
0 answers
127 views

Overall variance of a Dirichlet distribution

I have observations on proportion of individuals at different age groups. I'm doing some simulation experiments, and I need to introduce error in the observed data. For that, I'm simulating data from ...
  • 1
1 vote
0 answers
25 views

Marginal Cluster assignments for Dirichlet Process mixture model

I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
  • 1,016
2 votes
2 answers
765 views

How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution

I have derived the mean and variance of $\mu_j$ in Dirichlet distribution $\text{Dir}(\mu_1, \cdots, \mu_K|\alpha_1, \cdots, \alpha_k)$. On https://en.wikipedia.org/wiki/Dirichlet_distribution, it ...
  • 1,017
3 votes
2 answers
924 views

What is a non-informative choice of parameters for a Dirichlet distribution?

Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
  • 595
0 votes
1 answer
175 views

Uniform posterior on bounded space [duplicate]

In a particular Bayesian problem, I have encountered a choice of parameters that leads to a uniform posterior distribution. Given prior \begin{equation} p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\...
  • 595
1 vote
0 answers
65 views

How to model proportions with a hierarchical structure?

I have thinking about how to model proportions for a problem with hierarchical structure. In the problem, I have observations of users over multiple days, where each observation is a proportion of ...
  • 141
1 vote
2 answers
886 views

Softmax vs the Dirichlet distribution

As far as I understand one can in principle model the distribution over a set of $k$ categories using e.g.: the Dirichlet distribution A softmax model. As far as I can tell, both use $k$ parameters ...
  • 3,598
0 votes
1 answer
115 views

How does the uniform Dirichlet PDF integrate to 1?

For the uninformative 3-dimensional Dirichlet prior ${\rm Dir}(1, 1, 1)$, I understand that the probability density function (PDF) evaluates uniformly to 2, and the support are all three-dimensional ...

1
2 3 4 5 6