Questions tagged [dirichlet-process]

A family of stochastic processes whose realizations are probability distributions

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Estimating the likelihood of a Dirichlet process

I am not sure if what I'm trying to achieve makes sense or is even possible, but I'd like to do MLE on a Dirichlet process mixture model. My reasoning is the following: If we can write out the ...
user avatar
  • 273
0 votes
0 answers
13 views

Can we use Dirichlet process to simultaneously estimate the number of mixtures and component distribution of a Bernoulli mixture?

Suppose I have a random sample on a Bernoulli random variable $\{X_i\}_{i=1}^N$ generated from model $p=\sum_{k=1}^K\pi_kp_k$,where $p\equiv Pr(X=1)$ and $p_k\equiv Pr(X=1|k)$, and $\pi_k$ are the ...
user avatar
  • 1,682
0 votes
0 answers
19 views

Derivation of posterior distribution under Dirichlet prior distribution:

suppose that $\mathbf{y}=(y_1, y_2, \cdots, y_n)$ is a vector of $n$ observed sample points drawn from a mixture of $g$ components, and $\mathbf{z}=(z_1, z_2, \cdots, z_n)$ is a vector of latent ...
user avatar
0 votes
0 answers
23 views

Mixtures of Dirichlet multivariates or Dirichlet processes

I am exploring the properties of Dirichlet distributions and their parameters. When mixing two Dirichlet distributed random bivariates $$\mathbf{X}\equiv(X_1,X_2)\sim\text{Dir}(\alpha_1,\alpha_2)$$ ...
user avatar
0 votes
0 answers
24 views

Scoring rules for Dirichlet process mixtures

Let's say I do a sample-based fitting of a Dirichlet process model: $$ \begin{aligned} X_i &\sim f(x_i\mid \theta_i)\\ \theta_i &\sim G\\ G &\sim \text{DP}(\alpha, G_0) \end{aligned} $$ ...
user avatar
  • 173
2 votes
0 answers
113 views

Marginal distributions of the Indian Buffet Process

In the construction of the Indian Buffet Process we have that customer $n_1$ chooses $\mbox{Poisson}\left(\frac{\alpha}{1}\right)$ dishes, $n_2$ chooses $\mbox{Poisson}\left(\frac{\alpha}{2}\right) \...
user avatar
  • 151
1 vote
0 answers
64 views

Online Stochastic Variational Inference for Dirichlet Process Mixture Models

There's a 2013 NeurIPS paper I'm trying to understand, Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation. I have a few questions: Equation 2, which defines a ...
user avatar
0 votes
0 answers
558 views

Dirichlet Process vs Gaussian Process?

I'm studying the Dirichlet Process (DP) and looking to the Gaussian Process (GP), which I've had more experience with, to help make connections. The GP can receive $X_{train}$, $X_{test}$, and $Y_{...
user avatar
  • 1,700
3 votes
1 answer
159 views

Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?

The Chinese Restaurant Table Distribution describes the probability distribution for the number of non-empty tables in the Chinese Restaurant Process after $T$ customers have been seated. Specifically,...
user avatar
1 vote
1 answer
140 views

Validity of BIC for Dirichlet process mixture models

I am implementing clustering using Dirichlet process mixture models via scikit learn's Variational Bayesian Gaussian Mixture model. I arrived at the appropriate priors iteratively, and I am able to ...
user avatar
  • 63
2 votes
1 answer
144 views

Chinese Restaurant Process: Expected cardinality (number of customers) of each block (table)?

Short version of the question: The Chinese Restaurant Process defines a distribution over partitions of $[T] := \{1, ...., T\}$. What is the expected cardinality of the $t$th block, where $t \in \{1, ....
user avatar
3 votes
0 answers
107 views

A clarification in the original Dirichlet Process paper by Ferguson

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
user avatar
  • 1,282
1 vote
0 answers
30 views

Dirichlet Process vs Hierarchical Dirichlet Process: coupling among transitions on infinite HMM

I'm new to nonparametric Bayesian, and I am reading a paper about beam sampling for the infinite hidden Markov model. In the paper, it is mentioned that since there is no coupling among the ...
user avatar
  • 11
2 votes
0 answers
30 views

Unexpected zero on posterior density of Dirichlet process mixture

I was reading this notebook from the PyMC3 documentation about Dirichlet Process Mixtures and, on the last figure, the estimated density reaches almost zero for a particular value, despite the ...
user avatar
  • 2,536
4 votes
1 answer
282 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
user avatar
  • 468
1 vote
0 answers
23 views

Marginal Cluster assignments for Dirichlet Process mixture model

I am watching Tamara' Broderick video on Dirichlet Process mixture models where she talks about computing $p(z_n = k | z_1,z_2,..z_{n-1})$ at ardoun 16:06. The z's are drawn from $$\rho_1 \sim beta(...
user avatar
  • 886
1 vote
1 answer
36 views

Clarifying Wikipedia's Notation for Definition of a Dirichlet Process

Wikipedia gives the following definition of a Dirichlet process: What does the notation $H(B_i)$ and $X(B_i)$ mean? What does the notation $(X(B_1), ..., X(B_n))$ mean? My guess to the first ...
user avatar
0 votes
1 answer
52 views

What is the Dirichlet Proces Mixture Models posterior

I am trying to understand Dirichlet Process Mixture models. One of the videos I have been watching is by Tamara Broderick. I think it is a very good introductory video to Dirichlet Process mixture ...
user avatar
  • 886
1 vote
0 answers
44 views

Question about conditional probability in Dirichlet Process

Let $DP(\alpha_0, G_0)$ be a dirichlet process from which we sampled a distribution $G$. Furthermore we sample $\theta_1, ..., \theta_n$ from $G$ and let $A_1, ..., A_m$ be a partition on the support ...
user avatar
  • 2,743
3 votes
1 answer
58 views

Why are discrete random measures not dominated?

A statement I’ve seen without proof in many books and papers is that random probability measures obtained by normalizing a completely random measure, such as the Dirichlet process, are not dominated ...
user avatar
  • 33
0 votes
2 answers
130 views

Fitting/Inference for Dirichlet Process/CRP for clustering

Please excuse my ignorance on this topic, I don't have much experience in nonparametric Bayes. I read about Dirichlet process clustering and the Chinese Restaurant Process analogies. I think I ...
user avatar
0 votes
0 answers
41 views

Exchangabilty in CRF generative model for HDP

In the HDP Setting, the groups (or documents) are assumed exchangeable between them, and the samples (or words) within each group/topic are also exchangeable. Note the Setting chapter here However, ...
user avatar
  • 131
2 votes
0 answers
149 views

Graphical model of the Gaussian mixture: where is n?

TL;DR: Where are the occupation numbers in the Graphical model of the GMM? I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
user avatar
0 votes
1 answer
82 views

Understanding the sum in the stick breaking process

I am writing some R code for sampling from a Dirichlet process, but am having a hard time understanding how to take the final sum when there is a dirac delta ...
user avatar
5 votes
1 answer
285 views

Kernel density estimate vs Dirichlet process mixture

Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? ...
user avatar
  • 51
1 vote
0 answers
17 views

Is data modeled by dirichlet process mixture exchangeable?

Consider DPM model: $$ \begin{aligned} X_{i} | \phi_{i} & \sim F\left(x;\phi_{i}\right) \\ \phi_{1}, \phi_{2}, \cdots | & P \stackrel{iid}{\sim} P \\ P & \sim D P(\alpha G_0) \end{aligned} ...
user avatar
1 vote
0 answers
177 views

Why is Algorithm 8 of Neal (2000) a valid sampler?

I have been having difficulty understanding why Algorithm 8 of Neal (2000) is a valid sampler. I am looking for lecture notes that include a nice explanation of the proof. Does anyone know of any ...
user avatar
  • 123
1 vote
0 answers
131 views

Dirichlet Process Concentration Parameter - collapse at zero

Background I've implemented the blocked gibbs sampler for sampling from the posterior of a dirichlet process mixture model as described on p.552 of Bayesian Data Analysis, placing a Gamma prior on the ...
user avatar
1 vote
1 answer
126 views

Bayesian Inference: Prior in Chinese Restaurant Process

For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
user avatar
  • 13
2 votes
1 answer
265 views

Definition of the dirichlet process: what is the sequence of random variables

Reference material by Dr. Teh Definition Given a measureable set S, a base probability distribution H and a positive real number $\alpha$, the Dirichlet process $DP(H, \alpha)$ is a stochastic ...
user avatar
  • 2,049
1 vote
1 answer
274 views

Clusters keep switching in Gibbs sampling of Dirichlet Process Mixture Model

All the code and data for this question is on GitHub (stackexchange.R script). I've got multivariate Bernoulli data that I'd like to analyse using Bayesian Mixture ...
user avatar
4 votes
1 answer
134 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
user avatar
2 votes
1 answer
187 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
user avatar
0 votes
1 answer
1k views

Coding a simple Stick-Breaking Process in Python

I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python. I've got this ...
user avatar
1 vote
1 answer
292 views

Gibbs sampler for Dirichlet Process concentration parameter

I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
user avatar
  • 133
1 vote
0 answers
67 views

HDP: Gibbs sampler implementation

I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005). To estimate the model (let's call it iHDP) I need to ...
user avatar
  • 133
1 vote
0 answers
196 views

Dirichlet-Multinomial Distribution with many zero counts

Short version: Is there someway to make the dirchlet-multinomial distribution sensitive to the presence of zero counts? Long version: I am attempting model metric positions in musical data. You can ...
user avatar
1 vote
0 answers
100 views

How to interpret graphical model for Dirichlet process mixture for variational inference?

I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
user avatar
  • 885
2 votes
0 answers
73 views

How to actually draw a distribution from a given Dirichlet process?

I am wondering if there's an algorithm to actually draw a distribution from a given Dirichlet process. The closest thing I've came across is the stick-breaking construction of a Dirichlet process, but ...
user avatar
3 votes
1 answer
233 views

Understanding Dirichlet Process Mixtures

I have been reading a ton of stuff online and have a question about my understanding of Dirichlet Process Mixtures. First some basics on what I understand: Dirichlet Distribution: multivariate ...
user avatar
2 votes
1 answer
171 views

Explanation of Formal Definition of Dirichlet Process

I am reading about the Dirichlet process and I can understand the construction from Chinese restaurant process or stick-breaking process or Polya urn scheme. Now I am trying to understand why ...
user avatar
  • 1,643
3 votes
1 answer
611 views

Stick-breaking construction of Dirichlet process

In the stick-breaking construction of Dirichlet (let me base things on Sethuraman's construction - slide 6 of this) do we sample one $\phi$ vector from the base distribution $H$ and use it for ...
user avatar
  • 1,052
1 vote
1 answer
96 views

Why is the Dirichlet Process not a completely random measure?

A completely random measure assigns independent mass to nonintersecting subsets. I cannot figure, however, how the Dirichlet Process does not qualify as a CRM? Aren't the atoms all independent from ...
user avatar
1 vote
0 answers
140 views

Dirichlet process and clustering

How does clustering with a Dirichlet process as prior work? I am confused as to if the parameter $\alpha_i$ $\sim$ DP is found via clustering or is used to cluster. I undestrand how it can be used to ...
user avatar
  • 11
1 vote
0 answers
54 views

Clustering and Dirichlet process' parameter

I am reading a paper in which they describe a bayesian model in which the prior $a_i$ is defined as a Dirichlet Process (DP). They say: "We use a DP to find the optimal $a_i$ via clustering". Later on ...
user avatar
1 vote
0 answers
26 views

Q about closed form of distribution

Suppose you have a K boxes, each with associated probability $p_i$. You start putting balls in the boxes with probability of putting a ball in a box being proportional to $(n_i+1)\cdot p_i$ where $n_i$...
user avatar
  • 737
2 votes
0 answers
99 views

Applications of Hierarchical Dirichlet Process to Continuous Data

I read Yee Whye Teh et al.'s paper on Hierarchical Dirichlet Process. In section 5, they show sampling algorithm using base distribution H and data distribution F. One of their applications is HDP-...
user avatar
3 votes
1 answer
976 views

Predictive Density for Dirichlet Multinomial

I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is. In this tutorial (page 24), the predictive density is simple and something like "pseudo samples." However, ...
user avatar
  • 638
1 vote
0 answers
312 views

Collapsed Gibbs sampler on Hierarchical Dirichlet Process Mixture Model

I am trying to design a collapsed Gibbs sampler on a mixture model based on Hierarchical Dirichlet Process ($g\sim DP(\gamma, b)$, $\pi\sim DP(\alpha, g)$ ). Should I resample from the posterior of ...
user avatar
  • 884
0 votes
1 answer
298 views

In a Dirichlet process, can the base distribution be discrete?

Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete. 1) If the base distribution is continuous, drawing from it will get a new value (a new ...
user avatar
  • 197