Questions tagged [dirichlet-distribution]

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

Filter by
Sorted by
Tagged with
1
vote
2answers
30 views

Dirichlet distribution vs Multinomial distribution?

Both Dirichlet and multinomial distributions are distributions over vectors, and both Dirichlet and multinomial distributions are constrained so that all of the elements of these vectors sum to a ...
0
votes
0answers
12 views

log likelihood function causes huge differences with similar inputs

I am not sure whether there is a solution to this problem, but here goes. My problem is that in my function, I am taking log of a matrix and then taking its mean, and doing this for two very similar ...
0
votes
0answers
18 views

Posterior probability distribution from multinomial sample

I want to get the posterior from a multinomial sample and want to know if the following derivation is correct. Suppose when drawing (with replacement) a sample of $N$ balls from a urn with $K$ ...
0
votes
0answers
19 views

How to design a joint distribution of random variables when their summation is known

Suppose we have $D$ random variables $X_1, \ldots, X_D$, and $X_i\in[0,1]$ for $i=1,\cdots,D$. Can we somehow design a joint distribution $P(X_1, \ldots, X_D)$, such that $\sum_{i=1}^{D}X_i=s$, where $...
1
vote
1answer
39 views

Why use MCMC sampling when using conjugate priors?

I've been getting to grips with some Bayesian modelling, but one thing is confusing the heck out of me when I look at tutorials and worked-through problems online. I'm looking at a problem with a ...
0
votes
0answers
20 views

Question about possible typo in a tutorial about the stick-breaking model of the Dirichlet distribution

I am reading a tutorial on the Dirichlet distribution: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf and I think there is a typo in Step 2 of the stick-breaking model of ...
2
votes
0answers
23 views

Proving independence relationship

Let $X_1,X_2,X_3$ be continuous positive random variables satisfying $X_1+X_2+X_3<1$ and the following independence relations $$\frac{X_1}{X_1+X_2}\perp \!\!\!\perp \frac{X_3}{1-X_1-X_2}~ and$$ $$\...
1
vote
0answers
36 views

Proposal Function For Variables That Sum To 1 (Dirichlet Prior)

Recently I've been trying to use MCMC to infer a set of 50 random variables (species frequencies) that sum to 1 with the Metropolis-Hastings algorithm. However, the algorithm is not working well ...
0
votes
1answer
24 views

Normalized subvectors of Dirichlet, mutually independent?

Let$$X=(X_1,\cdots,X_k)\sim Dir(\alpha_1,\cdots,\alpha_k)$$ According to this reference, the independence of the two vectors, $$\bigg(\frac{X_1}{X_1+\cdots+X_j},\cdots,\frac{X_j}{X_1+\cdots+X_j}\...
1
vote
1answer
37 views

Inference on Dirichlet hyper-parameter

I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model. In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$...
0
votes
1answer
26 views

LDA alpha equivalent in structural topic model

I'm using an implementation of the structural topic model (stm), written in R using the stm package. I want to reduce the number of topics that are prevalent in ...
0
votes
1answer
29 views

If (x1,x2,…xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it?

If (x1,x2,...xn) follow a dirichlet distribution with all alphas=1, then I think xi~U(0,1)/summation[U(0,1)]. If yes, how to prove it? Note that xi are all positive and summation(xi)=1
-1
votes
1answer
47 views

I want to represent x1, x2, …, xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,… have the same pdf

I want to represent x1, x2, ..., xn (where their sum =1) by Dirichlet distribution. What alpha's should I select if x1, x2,...,xn have the same probability density function? all 0 < xi < 1. In ...
1
vote
1answer
72 views

could someone please give a concrete example to illustrate the Dirichlet distribution prior for bag-of-words?

I am aware of the notion of the Dirichlet distribution, a multivariate generalization of the beta distribution. To get parameters of the Dirichlet distribution prior for bag-of-words, this CMU ...
5
votes
1answer
75 views

Aggregation with an overlap: Dirichlet distribution

Suppose that we have $$(p_1,p_2,p_3,p_4)\sim Dirichlet(a_1,a_2,a_3,a_4),$$ where $p_4=1-p_1-p_2-p_3.$ When we add random variables for example, $p_1+p_2$ and $p_3+p_4$, the resulting distributions ...
4
votes
1answer
68 views

What is the name of this distribution?

I came across this: a categorical distribution with $K=10,000$ parameters (categories), and we take only few samples from this distribution, say $N=400$ (the point is $N < K$). Now, obviously, not ...
1
vote
2answers
189 views

Is it possible to fit a Dirichlet Regression to changing response variables?

A toy problem to illustrate my issue: We put a 100 people in a room with 10 candy bars. Each bar is different and has a different brand, flavor, size, color, etc. We ask each person in the group to ...
2
votes
1answer
107 views

Bayesian inference out of partial information - Dirichlet example

Suppose we have two coins $X_1$ and $X_2$. They are possibly biased and correlated coins. The heads probability of each coins is denoted by $p_1$ and $p_2$ which we don't know at the beginning. The ...
0
votes
0answers
14 views

Bayesian approaches to study multi-variate binomial case

I'm currently working on a machine learning problem and want to clarify my research question. Consider two coins $A$ and $B$, each of which respectively has Heads probability $p_A$ and $p_B$. ...
0
votes
0answers
19 views

Sum of elements in a Dirichlet random vector conditional on one element is greater than another

I'm working on a research project in machine learning but I was stuck at some point while developing a theory background of the project. Let $(X_1,X_2,X_3,X_4)\sim Dir(\alpha_1,\alpha_2,\alpha_3,\...
3
votes
2answers
210 views

Probability that a random variable is smaller than another in a random vector

Suppose that a random vector $X=(X_1,X_2,X_3)$ follows a Dirichlet distribution with a shape parameter $(a_1,a_2,a_3).$ What I want to calculate is the probability of $X_1>X_2$ and I want to ...
3
votes
2answers
155 views

Bayesian update for Beta distribution

I'm wondering how to find a posterior of a beta distribution when the "new information" is not an outcome of a binomial trial. Let $p$ be the probability of Head of a (biased) coin toss. As usual in ...
1
vote
1answer
55 views

Bayesian Inference: Prior in Chinese Restaurant Process

For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
1
vote
0answers
80 views

Treating missing data in making Bayesian inference

Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other. In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's ...
2
votes
0answers
30 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
1
vote
0answers
32 views

Modeling Bayesian inference using Dirichlet conjugate

I'm trying to formalize my research question and want to know whether the following set up makes any sense or not. Suppose there are two coins $a$ and $b$. Probability of tossing heads are given by $...
0
votes
0answers
57 views

Deriving conditional probability of bivariate bernoulli by using Dirichlet

While I was working on my research project, I found it difficult to derive a conditional probability from Dirichlet dist. Consider two Bernoulli trials that are possibly correlated with each other. ...
0
votes
0answers
37 views

Moments of the Dirichlet distribution [duplicate]

I was reading the Wikipedia article of the Dirichlet distribution which gives a general equation for the moments of a Dirichlet distributed random variable $X=(X_1,\cdots,X_K) \sim Dir(\boldsymbol{\...
0
votes
1answer
48 views

Bayesian inference using Dirichlet: muddled outcome case

In relation to my previous question (Bayesian inference for Beta distribution after an uncertain outcome), Suppose that $$(x_1,x_2,x_3)\sim Dirichlet(a_1,a_2,a_3)$$ and an associated Mutinoulli ...
0
votes
0answers
32 views

LDA: weight distributions of inferred documents

I have trained a two-topic Latent Dirichlet Allocation (LDA) model on a corpus and I am now inferring on a test corpus (the nature of the corpus is irrelevant). During inference, for each new document ...
1
vote
0answers
35 views

Difference between two dimensions sampled from Dirichlet distribution

Say I'm doing Bayesian inference on a Dirichlet-Multinomial model: $$ x \in [1,2,3]; \\ x \sim Multinomial(p_1, p_2, p_3); \\ p_1, p_2, p_3 \sim Dirichlet(\alpha_1, \alpha_2, \alpha_3); \\ \alpha_n =...
3
votes
1answer
184 views

From beta distribution to Dirichlet: Estimation of the concentrantion parameters

Searching at least 3 hours about the connection between beta distribution and dirichlet. My problem is: I have a collection of random variables $X_i \sim Beta(a_i, b_i)$. The parameters $a_i$ and $...
2
votes
1answer
54 views

Applying de Finetti's representation theorem to Dirichlet distribution

Let's begin from the de Finetti–Hewitt–Savage theorem: for an exchangeable sequence of random variables we can always write $$ p(x_1, x_2,\cdots) = \int \prod p(x_i | L) P(dL) $$ where $L$ is a latent ...
2
votes
1answer
108 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
0
votes
0answers
50 views

Multinomial-dirichlet with fractional counts

Suppose a lepidopterologist wants to estimate the relative proportions of three different species of butterfly. They go out into the field and count $N$ butterflies and record the number of each ...
0
votes
0answers
85 views

Is fair to compare Dirichlet to a Multivariate Beta regression?

I am conducting some analysis on my data I found a strange behavior and would greatly appreciate some guidance or suggestions. I am trying to investigate the effect of a categorical variable (cl) to ...
1
vote
0answers
128 views

Dirichlet Multinomial Posterior Predictive Distribution for Language Model

I have been trying to teach myself about Bayesian analysis, and whilst I have been through the theory several times, I am struggling to actually apply it. I have found some questions online to ...
0
votes
1answer
424 views

Coding a simple Stick-Breaking Process in Python

I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python. I've got this ...
5
votes
1answer
172 views

How is the mode in Dirichlet-Multinomial calculated?

The mode in Dirichlet-Multinomial is $$ \mathrm{Mode}(\pi_i) = \frac{\alpha_i + x_i - 1}{\sum_{j=1}^k (\alpha_j + x_j -1)} $$ Could you point out how is it calculated please? What is the importance ...
0
votes
1answer
96 views

What are the possible estimates of the parameters of the multinomial distribution?

The expected value of the parameters of the multinomial distribution (taking into account the Dirichlet prior $D(\alpha)$ and the posterior Dirichlet-Multinomial) is: $\pi_i = α_i+ x_i / \sum_{j} α_j+...
1
vote
1answer
783 views

How to use the Dirichlet prior for estimating the multinomial parameters? [closed]

I know that the multinomial distribution gives the likelihood of some vector D of occurrences to happen given a probability vector (parameters) P' i.e. P(D|P'). Now with a Dirichlet prior we are ...
1
vote
0answers
152 views

Understanding Multivariate Logistic Normal pdf

I'm trying to understand the Multivariate Logistic Normal distribution, in order to plot its pdf and compare it with a Dirichlet distribution. I believe I can follow the pdf derivation for the ...
0
votes
1answer
337 views

Bayesian posterior pmf for weighted dice with uniform prior

We want to find posterior probability mass function for dice tossing with uniform prior. We are interested in rolling of weighted dice. The outcome is 1,2,...,6. We assume that prior probability ...
2
votes
0answers
45 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
3
votes
1answer
159 views

weighted sum of posterior Dirichlet distributions

I have the following distribution: $q(\vec\theta) = \frac{\sum_k \alpha_k}{\sum_k \beta_k \alpha_k} (\sum_k\theta_k\beta_k) \frac{\Gamma(\sum_k \alpha_k)}{\prod_k\Gamma(\alpha_k)} \prod_k \theta_{k}^{...
0
votes
1answer
46 views

Definition of distribution conditioned on both a categorical and Dirichlet prior

If we have a conditional categorical distribution, with unknown parameters, we can represent with a table, as in the example below: \begin{align*} &z \quad P(z|\theta)\\ &0 \quad \theta_0\\ &...
3
votes
3answers
127 views

Distribution of *conditional* frequencies when frequencies follow a Dirichlet distribution

Context: we have a large number of individuals characterized by two binary traits; call these $T$ with values $\{0,1\}$, and $T'$ with values $\{0',1'\}$. So there are four types of individuals: $00'$,...
0
votes
0answers
56 views

Recovering $\theta$ in Dirichlet-Multinomial (Polya) distribution

I'm working on Latent Dirichlet Allocation with Collapsed Gibbs Sampling. LDA has two Dirichlet-Multinomial distribution and one of them is a document-topic distribution that determines the ...
1
vote
0answers
176 views

Marginal Likelihood of Multinomial Dirichlet model

To find the marginal likelihood of the multinomial Dirichlet model, I tried the following: $$\int_\theta p(N|\theta)p(\theta)d\theta=\frac{n!}{n_1!...n_K!}\frac{\Gamma(\sum_{k=1}^K\alpha_k)}{\Pi_{k=1}^...
1
vote
0answers
175 views

How to specify a zero-inflated Dirichlet model in JAGS/BUGS

There was a recent publication discussing the advantages of the zero-inflated dirichlet for microbiome count data which is compositional (you are modeling a matrix of species relative abundance data ...

1
2 3 4 5