Questions tagged [dirichlet-process]

A family of stochastic processes whose realizations are probability distributions

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Looking for a sample that proves that KDE behaves worse than the Dirichlet Process

I am trying to find an example that clearly shows that the kernel density estimator does worse than the Dirichlet process in terms of estimating the distribution of a sample. But eventually, I always ...
0
votes
0answers
20 views

Exchangabilty in CRF generative model for HDP

In the HDP Setting, the groups (or documents) are assumed exchangeable between them, and the samples (or words) within each group/topic are also exchangeable. Note the Setting chapter here However, ...
2
votes
0answers
70 views

Graphical model of the Gaussian mixture: where is n?

TL;DR: Where are the occupation numbers in the Graphical model of the GMM? I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
0
votes
1answer
23 views

Understanding the sum in the stick breaking process

I am writing some R code for sampling from a Dirichlet process, but am having a hard time understanding how to take the final sum when there is a dirac delta ...
3
votes
1answer
70 views

Kernel density estimate vs Dirichlet process mixture

Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? ...
1
vote
0answers
9 views

Is data modeled by dirichlet process mixture exchangeable?

Consider DPM model: $$ \begin{aligned} X_{i} | \phi_{i} & \sim F\left(x;\phi_{i}\right) \\ \phi_{1}, \phi_{2}, \cdots | & P \stackrel{iid}{\sim} P \\ P & \sim D P(\alpha G_0) \end{aligned} ...
0
votes
0answers
4 views

Dirchlet Process Mixture of Multinomials

The Dirichlet Process Mixture of Gaussians has been well studied and shown to work. I have never seen DPM of Multinomials and tried to implement one. What I notice is the likelihood tends to dominate ...
1
vote
0answers
48 views

Why is Algorithm 8 of Neal (2000) a valid sampler?

I have been having difficulty understanding why Algorithm 8 of Neal (2000) is a valid sampler. I am looking for lecture notes that include a nice explanation of the proof. Does anyone know of any ...
1
vote
0answers
74 views

Dirichlet Process Concentration Parameter - collapse at zero

Background I've implemented the blocked gibbs sampler for sampling from the posterior of a dirichlet process mixture model as described on p.552 of Bayesian Data Analysis, placing a Gamma prior on the ...
1
vote
1answer
55 views

Bayesian Inference: Prior in Chinese Restaurant Process

For the Chinese restaurant process, as used in Dirichlet Process mixture models, we have a prior that data point i belongs to cluster j, where c is an indicator. n represents the total number of data ...
1
vote
1answer
91 views

Definition of the dirichlet process: what is the sequence of random variables

Reference material by Dr. Teh Definition Given a measureable set S, a base probability distribution H and a positive real number $\alpha$, the Dirichlet process $DP(H, \alpha)$ is a stochastic ...
0
votes
1answer
80 views

Clusters keep switching in Gibbs sampling of Dirichlet Process Mixture Model

All the code and data for this question is on GitHub (stackexchange.R script). I've got multivariate Bernoulli data that I'd like to analyse using Bayesian Mixture ...
4
votes
1answer
80 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
2
votes
1answer
109 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
0
votes
1answer
434 views

Coding a simple Stick-Breaking Process in Python

I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python. I've got this ...
1
vote
1answer
169 views

Gibbs sampler for Dirichlet Process concentration parameter

I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
1
vote
0answers
58 views

HDP: Gibbs sampler implementation

I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005). To estimate the model (let's call it iHDP) I need to ...
1
vote
0answers
101 views

Dirichlet-Multinomial Distribution with many zero counts

Short version: Is there someway to make the dirchlet-multinomial distribution sensitive to the presence of zero counts? Long version: I am attempting model metric positions in musical data. You can ...
1
vote
0answers
72 views

How to interpret graphical model for Dirichlet process mixture for variational inference?

I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
2
votes
0answers
65 views

How to actually draw a distribution from a given Dirichlet process?

I am wondering if there's an algorithm to actually draw a distribution from a given Dirichlet process. The closest thing I've came across is the stick-breaking construction of a Dirichlet process, but ...
3
votes
1answer
152 views

Understanding Dirichlet Process Mixtures

I have been reading a ton of stuff online and have a question about my understanding of Dirichlet Process Mixtures. First some basics on what I understand: Dirichlet Distribution: multivariate ...
1
vote
1answer
99 views

Explanation of Formal Definition of Dirichlet Process

I am reading about the Dirichlet process and I can understand the construction from Chinese restaurant process or stick-breaking process or Polya urn scheme. Now I am trying to understand why ...
3
votes
1answer
352 views

Stick-breaking construction of Dirichlet process

In the stick-breaking construction of Dirichlet (let me base things on Sethuraman's construction - slide 6 of this) do we sample one $\phi$ vector from the base distribution $H$ and use it for ...
1
vote
1answer
86 views

Why is the Dirichlet Process not a completely random measure?

A completely random measure assigns independent mass to nonintersecting subsets. I cannot figure, however, how the Dirichlet Process does not qualify as a CRM? Aren't the atoms all independent from ...
1
vote
0answers
99 views

Dirichlet process and clustering

How does clustering with a Dirichlet process as prior work? I am confused as to if the parameter $\alpha_i$ $\sim$ DP is found via clustering or is used to cluster. I undestrand how it can be used to ...
1
vote
0answers
48 views

Clustering and Dirichlet process' parameter

I am reading a paper in which they describe a bayesian model in which the prior $a_i$ is defined as a Dirichlet Process (DP). They say: "We use a DP to find the optimal $a_i$ via clustering". Later on ...
1
vote
0answers
25 views

Q about closed form of distribution

Suppose you have a K boxes, each with associated probability $p_i$. You start putting balls in the boxes with probability of putting a ball in a box being proportional to $(n_i+1)\cdot p_i$ where $n_i$...
2
votes
0answers
79 views

Applications of Hierarchical Dirichlet Process to Continuous Data

I read Yee Whye Teh et al.'s paper on Hierarchical Dirichlet Process. In section 5, they show sampling algorithm using base distribution H and data distribution F. One of their applications is HDP-...
2
votes
1answer
557 views

Predictive Density for Dirichlet Multinomial

I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is. In this tutorial (page 24), the predictive density is simple and something like "pseudo samples." However, ...
1
vote
0answers
257 views

Collapsed Gibbs sampler on Hierarchical Dirichlet Process Mixture Model

I am trying to design a collapsed Gibbs sampler on a mixture model based on Hierarchical Dirichlet Process ($g\sim DP(\gamma, b)$, $\pi\sim DP(\alpha, g)$ ). Should I resample from the posterior of ...
0
votes
1answer
209 views

In a Dirichlet process, can the base distribution be discrete?

Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete. 1) If the base distribution is continuous, drawing from it will get a new value (a new ...
0
votes
1answer
38 views

functionals of Dirichlet process mixture

I want to solve (or simulate) /int f(x)G(dx) where f is a function(or phrase of x) and G is DPM(Dirichlet Process mixture prior). Muliere (1998) approximate this ...
2
votes
0answers
575 views

Dirichlet Process Posterior

I'm reading through a tutorial on the Dirichlet process (http://www.stats.ox.ac.uk/~teh/research/npbayes/Teh2010a.pdf) and have a small question. Given a draw from a Dirichlet process $G\sim\textsf{DP}...
3
votes
0answers
179 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
0
votes
0answers
32 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
1
vote
0answers
347 views

Need a good book on Dirichlet Process

Was curious if there is any good book on Dirichlet Process (DP). I'm reading the DBA 3 by Andrew Gelman and I didn't understand it well enough. I was curious if there is a book that go into it a bit ...
1
vote
0answers
123 views

A Problem with Dirichlet process mixture model (DPMM)

I know that DPMM is a non-parametric Bayesian clustering model which can automatically determine the number of clusters based on data. However, which kind of clusters does the model find? For example,...
2
votes
0answers
391 views

Gibbs sampling in the Hierarchical Dirichlet Process

For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
1
vote
0answers
376 views

Understanding Bayesian Histogram

I'm reading about Dirichlet Process Models in "Bayesian Data Analysis" by Gelman et. al. To motivate the idea, they start with a section on Bayesian histograms. I am a little confused about their ...
3
votes
1answer
843 views

What does the base distribution of the Dirichlet Process mean?

So far I only really understand the Dirichlet Process through its various metaphors. For the Polya Urn scheme, my understanding is that the "base distribution" is the original distribution of colors ...
1
vote
1answer
149 views

Nonparametric topic modeling: hierarchical dirichlet vs. Indian buffet?

The hierarchical dirichlet process (Teh 2005) allows you to discover unlimited topics to describe a document. An alternative process, the Indian Buffet process (Griffiths 2011) is another ...
4
votes
2answers
542 views

Posterior of parameter for Chinese Restaurant Process

I have a Chinese Restaurant Process with (unknown) concentration parameter $\theta$. After $n$ customers have been seated, I observe the number of non-empty tables and the number of people at each ...
3
votes
1answer
280 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
1
vote
1answer
68 views

Dirichlet Process with known mean

A Dirichlet process $\operatorname{DP}(\alpha,G_0)$ can be thought of as a distribution of distributions. I would like an object similar to a Dirichlet process, but which has support only on ...
5
votes
1answer
323 views

How can we convert values proportional to probabilities to Bernoulli probabilities?

According to Wikipedia, the parameter in a Bernoulli distribution should be $0<p<1$. I am reading this famous paper proposing Hierarchical Dirichlet Process, and on page 1580, A.6 and the ...
0
votes
0answers
67 views

Given a pdf which is a mixture of Gaussians, how do I infer the position (mean), variance, and number of Gaussians?

I have the following data, which when plotted as a histogram, are a mixture of Gaussians: I would like to write an algorithm that would infer: (1) the number of "peaks" or normal distributions in ...
2
votes
2answers
176 views

How does this Sampler work for the Concentration parameter of Dirichlet Process?

I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ ...
3
votes
0answers
88 views

Bayesian nonparametrics vs model selection using Minimum Message Length

As we know mixture models are important tools in density estimation and in general in statistical machine learning. I have always used nonparametric Bayesian mixture models to avoid the problem of ...
1
vote
1answer
164 views

Notation for base distribution in Dirichlet process

I have a (hopefully simple) question about notation when defining a DP. I have read a lot of papers on DPs, but this is not clear to me, or at least I have not noticed a convention. Say that I am ...
0
votes
1answer
171 views

Sampling Concentration Parameter of DP according to Escobar and West

I am reading Escobar&West paper and in particular am interested in their Gibbs sampler for the concentration parameter of Dirichlet Process (eq 13, eq 14). Given this, $$p(\alpha,\eta|k)\propto p(...