Questions tagged [dirichlet-process]
A family of stochastic processes whose realizations are probability distributions
138
questions
4
votes
2
answers
361
views
Dirichlet distribution with correlated components?
I am working with models that use Dirichlet distributions. However, I want to account for correlations between components. If this question is a duplicate, I'd also appreciate any pointers to the ...
0
votes
0
answers
31
views
Stick-breaking construction of Dirichlet distribution vs Dirichlet process
Let $F_0$ be some probability measure and $\alpha > 0$ be the concentration parameter. I can draw a random distribution from $F\sim \mathrm{DP}(\alpha, F_0)$ using the stick-breaking construction:
\...
- 143
0
votes
0
answers
53
views
Expected value (and variance) of a Dirichlet Process
Suppose I have a measure $G$ that follows a Dirichlet Process,
$$G \sim DP(H_0,\alpha)$$
where $H_0$ is some base measure. Is there a closed form solution for the expected value of $G$?
0
votes
0
answers
8
views
Choice of base measure in nCRP (for validity and computation)
I'm trying to apply the nested Chinese restaurant process (nCRP) for structure learning. nCRP is a nested extension of CRP, with each table in CRP uniquely pointing to the restaurant on the next level....
- 1
0
votes
0
answers
40
views
Dirichlet Mixture Model manually
I am implementing the Dirichlet Mixture Model using the EM algorithm in R, but am experiencing issues with the results. I generated two binomial distributions with fractions of (70%, 30%) and means of ...
0
votes
0
answers
15
views
If a scalar is added to a Dirichlet process with a normal base distribution, is the resulting process also Dirichlet?
Let $\theta_i | G \sim G$ and $G|\alpha \sim DP(\alpha, G_0)$ with $G_0 \sim N(0,\sigma^2)$. For a scalar k, does $\theta_i + k | G' \sim G'$ with $G'|\alpha \sim DP(\alpha, G'_0)$ and $G'_0 \sim N(k,\...
- 86
1
vote
1
answer
97
views
Changing scan order of Gibbs Sampler on each iteration
I'm implementing an algorithm that requires the use of Gibbs Sampling and, due to the nature of the way I store the values, it would be efficient to change the order of the updates on each component ...
- 31
1
vote
0
answers
47
views
Dirichlet Process posterior with partially observed data
Suppose I dipose of a set of independant observed couples $(x_1,y_1),...,(x_N, y_N)$ from a joint distribution $P(x,y)$. Furthermore, I suppose that the random distribution $P$ as a Dirichlet prior
$P\...
- 11
0
votes
0
answers
22
views
Are Polya urn (CRP) and stick-breaking clusters interchangeable in posterior analysis?
In sampling a DP model, it's more space-efficient to only keep data related to active clusters (clusters with data). Under a CRP model, if I want to do posterior predictive sampling, it's ...
- 245
1
vote
0
answers
105
views
Stick-breaking: break sticks of decreasing lengths
The stick-breaking construction used for Dirichlet Processes can create an infinite sequence of probabilities $ \boldsymbol{\pi} $ (stick lengths) that sum to 1 via the following formulae:
$\nu_i \sim ...
- 83
1
vote
1
answer
116
views
Understanding the blocked gibbs sampler for Dirichlet process
I've always implemented DP MM's using chinese restaurant process, which necessitates sequential sampling of cluster assignments (as cluster weights depend on current number of observations in cluster, ...
- 245
1
vote
1
answer
58
views
Let $H$ be the base distribution of a Dirichlet process. How is this process well-defined in case $H(B_1) = 0$?
I have read that the parameters of Dirichlet distribution must be strictly positive.
The Dirichlet distribution of order $K \geq 2$ with parameters $\alpha_{1}, \ldots, \alpha_{K} \color{blue}{> 0}...
- 403
0
votes
0
answers
257
views
Estimating the likelihood of a Dirichlet process
I am not sure if what I'm trying to achieve makes sense or is even possible, but I'd like to do MLE on a Dirichlet process mixture model. My reasoning is the following:
If we can write out the ...
- 303
0
votes
0
answers
31
views
Can we use Dirichlet process to simultaneously estimate the number of mixtures and component distribution of a Bernoulli mixture?
Suppose I have a random sample on a Bernoulli random variable $\{X_i\}_{i=1}^N$ generated from model $p=\sum_{k=1}^K\pi_kp_k$,where $p\equiv Pr(X=1)$ and $p_k\equiv Pr(X=1|k)$, and $\pi_k$ are the ...
- 2,516
0
votes
0
answers
74
views
Mixtures of Dirichlet multivariates or Dirichlet processes
I am exploring the properties of Dirichlet distributions and their parameters. When mixing two Dirichlet distributed random bivariates
$$\mathbf{X}\equiv(X_1,X_2)\sim\text{Dir}(\alpha_1,\alpha_2)$$
...
0
votes
0
answers
31
views
Scoring rules for Dirichlet process mixtures
Let's say I do a sample-based fitting of a Dirichlet process model:
$$
\begin{aligned}
X_i &\sim f(x_i\mid \theta_i)\\
\theta_i &\sim G\\
G &\sim \text{DP}(\alpha, G_0)
\end{aligned}
$$
...
- 245
2
votes
0
answers
123
views
Marginal distributions of the Indian Buffet Process
In the construction of the Indian Buffet Process we have that customer $n_1$ chooses $\mbox{Poisson}\left(\frac{\alpha}{1}\right)$ dishes, $n_2$ chooses $\mbox{Poisson}\left(\frac{\alpha}{2}\right) \...
- 133
1
vote
0
answers
86
views
Online Stochastic Variational Inference for Dirichlet Process Mixture Models
There's a 2013 NeurIPS paper I'm trying to understand, Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation. I have a few questions:
Equation 2, which defines a ...
- 915
2
votes
0
answers
882
views
Dirichlet Process vs Gaussian Process?
I'm studying the Dirichlet Process (DP) and looking to the Gaussian Process (GP), which I've had more experience with, to help make connections.
The GP can receive $X_{train}$, $X_{test}$, and $Y_{...
- 2,377
3
votes
1
answer
310
views
Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?
The Chinese Restaurant Table Distribution describes the probability distribution for the number of non-empty tables in the Chinese Restaurant Process after $T$ customers have been seated. Specifically,...
- 915
1
vote
1
answer
271
views
Validity of BIC for Dirichlet process mixture models
I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
- 63
2
votes
1
answer
408
views
Chinese Restaurant Process: Expected cardinality (number of customers) of each block (table)?
Short version of the question: The Chinese Restaurant Process defines a distribution over partitions of $[T] := \{1, ...., T\}$. What is the expected cardinality of the $t$th block, where $t \in \{1, ....
- 915
3
votes
0
answers
120
views
A clarification in the original Dirichlet Process paper by Ferguson
I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
- 1,542
1
vote
0
answers
44
views
Dirichlet Process vs Hierarchical Dirichlet Process: coupling among transitions on infinite HMM
I'm new to nonparametric Bayesian, and I am reading a paper about beam sampling for the infinite hidden Markov model. In the paper, it is mentioned that since there is no coupling among the ...
- 11
2
votes
0
answers
39
views
Unexpected zero on posterior density of Dirichlet process mixture
I was reading this notebook from the PyMC3 documentation about Dirichlet Process Mixtures and, on the last figure, the estimated density reaches almost zero for a particular value, despite the ...
- 2,664
4
votes
1
answer
647
views
Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior
I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
- 538
1
vote
0
answers
42
views
Marginal Cluster assignments for Dirichlet Process mixture model
I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
- 1,096
1
vote
1
answer
49
views
Clarifying Wikipedia's Notation for Definition of a Dirichlet Process
Wikipedia gives the following definition of a Dirichlet process:
What does the notation $H(B_i)$ and $X(B_i)$ mean?
What does the notation $(X(B_1), ..., X(B_n))$ mean?
My guess to the first ...
- 915
1
vote
1
answer
152
views
What is the Dirichlet Proces Mixture Models posterior
I am trying to understand Dirichlet Process Mixture models. One of the videos I have been watching is by Tamara Broderick. I think it is a very good introductory video to Dirichlet Process mixture ...
- 1,096
1
vote
0
answers
51
views
Question about conditional probability in Dirichlet Process
Let $DP(\alpha_0, G_0)$ be a dirichlet process from which we sampled a distribution $G$.
Furthermore we sample $\theta_1, ..., \theta_n$ from $G$ and let $A_1, ..., A_m$ be a partition on the support ...
- 2,931
3
votes
1
answer
72
views
Why are discrete random measures not dominated?
A statement I’ve seen without proof in many books and papers is that random probability measures obtained by normalizing a completely random measure, such as the Dirichlet process, are not dominated ...
- 33
0
votes
2
answers
274
views
Fitting/Inference for Dirichlet Process/CRP for clustering
Please excuse my ignorance on this topic, I don't have much experience in nonparametric Bayes. I read about Dirichlet process clustering and the Chinese Restaurant Process analogies. I think I ...
0
votes
0
answers
44
views
Exchangabilty in CRF generative model for HDP
In the HDP Setting, the groups (or documents) are assumed exchangeable between them, and the samples (or words) within each group/topic are also exchangeable.
Note the Setting chapter here
However, ...
- 131
4
votes
0
answers
275
views
Graphical model of the Gaussian mixture: where is n?
TL;DR: Where are the occupation numbers in the Graphical model of the GMM?
I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
- 479
0
votes
1
answer
138
views
Understanding the sum in the stick breaking process
I am writing some R code for sampling from a Dirichlet process, but am having a hard time understanding how to take the final sum when there is a dirac delta ...
- 97
7
votes
1
answer
721
views
Kernel density estimate vs Dirichlet process mixture
Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? ...
- 71
1
vote
0
answers
28
views
Is data modeled by dirichlet process mixture exchangeable?
Consider DPM model:
$$
\begin{aligned} X_{i} | \phi_{i} & \sim F\left(x;\phi_{i}\right) \\ \phi_{1}, \phi_{2}, \cdots | & P \stackrel{iid}{\sim} P \\ P & \sim D P(\alpha G_0) \end{aligned}
...
- 241
1
vote
1
answer
311
views
Why is Algorithm 8 of Neal (2000) a valid sampler?
I have been having difficulty understanding why Algorithm 8 of Neal (2000) is a valid sampler.
I am looking for lecture notes that include a nice explanation of the proof. Does anyone know of any ...
- 123
1
vote
0
answers
164
views
Dirichlet Process Concentration Parameter - collapse at zero
Background
I've implemented the blocked gibbs sampler for sampling from the posterior of a dirichlet process mixture model as described on p.552 of Bayesian Data Analysis, placing a Gamma prior on the ...
1
vote
1
answer
204
views
Bayesian Inference: Prior in Chinese Restaurant Process
For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
- 13
3
votes
1
answer
469
views
Definition of the dirichlet process: what is the sequence of random variables
Reference material by Dr. Teh
Definition
Given a measureable set S, a base probability distribution H and a
positive real number $\alpha$, the Dirichlet process $DP(H, \alpha)$
is a stochastic ...
- 2,386
1
vote
1
answer
412
views
Clusters keep switching in Gibbs sampling of Dirichlet Process Mixture Model
All the code and data for this question is on GitHub (stackexchange.R script).
I've got multivariate Bernoulli data that I'd like to analyse using Bayesian Mixture ...
- 688
5
votes
1
answer
187
views
Dirichlet process mixture modelling for a Gaussian likelihood
Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
- 231
2
votes
1
answer
247
views
Clarifying Dirichlet Process Mixture Probability Terms
Suppose I have a Dirichlet Process Mixture model defined as follows:
$\alpha \sim G(a,b)\\
\pi|\alpha \sim \text{Dir}(\alpha)\\
z|\pi \sim \text{Cat}(\pi)\\
$
where $G$ is just a standard Gamma ...
- 455
0
votes
2
answers
1k
views
Coding a simple Stick-Breaking Process in Python
I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python.
I've got this ...
- 149
1
vote
1
answer
351
views
Gibbs sampler for Dirichlet Process concentration parameter
I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
- 153
1
vote
0
answers
85
views
HDP: Gibbs sampler implementation
I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005).
To estimate the model (let's call it iHDP) I need to ...
- 153
1
vote
0
answers
317
views
Dirichlet-Multinomial Distribution with many zero counts
Short version: Is there someway to make the dirchlet-multinomial distribution sensitive to the presence of zero counts?
Long version:
I am attempting model metric positions in musical data.
You can ...
- 131
1
vote
0
answers
125
views
How to interpret graphical model for Dirichlet process mixture for variational inference?
I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
- 1,015
2
votes
0
answers
76
views
How to actually draw a distribution from a given Dirichlet process?
I am wondering if there's an algorithm to actually draw a distribution from a given Dirichlet process. The closest thing I've came across is the stick-breaking construction of a Dirichlet process, but ...
- 21