12 votes

Can someone give a simple guide of Dirichlet process clustering?

What is the difference between (Dirichlet) distribution and (Dirichlet) process? The difference between a Dirichlet distribution and a Dirichlet process is perhaps easier to understand when you ...
6 votes

Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters

To demonstrate a solution to this hyperprior problem, I implemented an hierarchical gamma-Dirichlet-multinomial model in PyMC3. The gamma prior for the Dirichlet is specified and sampled per Ted ...
  • 61
5 votes

Understanding and Implementing a Dirichlet Process model

What is the difference between DP and CRP? The Chinese Restaurant Process (CRP) is a distribution over partitions of integers. The connection to the Dirichlet Process (DP) exists thanks to De Finetti'...
5 votes
Accepted

What does $\in$ mean vs $=$ in probability? What does $d\phi$ mean?

This is indeed rather confusing: the notation $d\phi$ stands for an infinitesimal measurable set located around $\phi$. As in standard measure theory settings with Leibniz's $dx$. It can thus be used ...
  • 94.5k
5 votes
Accepted

How can we convert values proportional to probabilities to Bernoulli probabilities?

Since $p(1)=p$ and $p(0)=1-p$ are both proportional to a known expression* (the unscaled probabilities, $u(i)=c.p(i)$, with the same unknown constant of proportionality, $c$) and you know the $p(i)$ ...
  • 264k
5 votes
Accepted

Predictive Density for Dirichlet Multinomial

The first slide applies the general result of the second slide to a case when $$y=\overbrace{(0,\ldots,0,1,0,\ldots,0)}^{1\text{ at }\kappa\text{th position}}$$ with different notations, since [in the ...
  • 94.5k
4 votes
Accepted

Chinese Restaurant Process

You need to be very precise about what $c$ and $n$ are. Let $c_i$ be the number of customers at table $i$. Let $n$ be the customer number, i.e. $n-1$ clients are already seated at all tables. Let $...
4 votes
Accepted

Truncated Dirichlet process vs Dirichlet distribution

You have two very different priors on $\pi$, so why would you expect the same posterior? Let's think about the nature of $T$ in your presentation above: In 1: Dirichlet Distribution, $T$ is a ...
  • 4,737
4 votes
Accepted

What does it mean to integrate over a random measure?

Denote by $\mathcal{M}$ a measurable space of probability measures, containing the realisations of the Dirichlet process. The random probability measure $G$ is a measurable function $$ G : \omega \...
  • 769
4 votes
Accepted

Escobar and West Sampler for Dirichlet Process Parameters

The (conditional) marginal distribution can be computed from the (conditional) joint distribution: $$ p(\alpha|k) = \int p(\alpha,\eta|k)\,d\eta. $$ Therefore, if $$ p(\alpha,\eta|k) = c\,p(\alpha)\,\...
  • 3,011
4 votes
Accepted

Dirichlet Process Posterior

The posterior base distribution must be a probability distribution. Let $M$ be the concentration parameter. For any $M$ $$\frac{\alpha}{M}H+\frac{1}{M}\sum_i \delta_{\theta_i}(\cdot)$$ is a finite ...
4 votes
Accepted

Understanding Dirichlet Process Mixtures

Probably it is better to describe how one would generate data from a Dirichlet process mixture. Each line is understood to be conditional on all lines above it. Sample $P \sim \operatorname{DP}(\...
  • 8,102
4 votes

Dirichlet process mixture modelling for a Gaussian likelihood

Chinese restaurant process Let's define this problem using the Chinese Restaurant Process (CRP) formulation of the Dirichlet Process (DP), which can be summarized as follows (from Gershman et al., ...
  • 3,172
4 votes
Accepted

Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?

We can apply the Lyapounov CLT to show that $K_T$ is normal for large $T$. Let $$ K_T = \sum_{i=1}^T I(\text{new table at $i$}) = \sum_{i=1}^T Z_i. $$ Then it is well-known that $Z_i \stackrel{\text{...
  • 8,102
3 votes

Dirichlet process

A realization from a DP is a discrete distribution $G$, not a Dirichlet distribution. Basically a DP is a distribution from which you sample distributions. If you ever studied how you can sample ...
3 votes
Accepted

Clustering methods for unknown number of clusters

which other clustering methods (unsupervised classification) can I try for this problem? For instance, parametric ones: you can fit a Gaussian Mixture Model by Expectation Maximization or ...
  • 2,706
3 votes

Is there a Bayesian approach to density estimation

For density estimation purposes what you need is not $\theta_{n+1}|x_{1},\ldots,x_{n}$. The formula in notes $\theta_{n+1}|\theta_{1},\ldots,\theta_{n}$ reffers to the predictive distribution of the ...
  • 490
3 votes

Is there a Bayesian approach to density estimation

Since you want a bayesian approach, you need to assume some prior knowledge about the thing you want to estimate. This will be in the form of a distribution. Now, there's the issue that this is now a ...
  • 416
3 votes
Accepted

What does the base distribution of the Dirichlet Process mean?

Let $$ G \sim \textsf{DP}(\alpha, H) $$ which says that the random distribution $G$ is itself distributed according to the Dirichlet Process with concentration parameter $\alpha$ and base ...
  • 3,011
3 votes
Accepted

Definition of the dirichlet process: what is the sequence of random variables

The notes you reference have a splendid, clear, insightful discussion of this definition, its meaning, and its application--thank you for bringing them to our attention. Because the job of ...
  • 297k
2 votes

Understanding the derivation of an equation in LDA modeling

Actually it was not "removed", the symbol does not mean equal! It means that the equation is proportional to the other. The denominator of the function is used to normalize the value and obtain a ...
  • 545
2 votes

Monte Carlo computation of expectation when there is dirac delta

The problem in the first part of your question is wrongly formulated given that the conditional density does not exist. According to slide 44, what you are interested in is the Posterior Predictive ...
  • 81
2 votes

Understanding and Implementing a Dirichlet Process model

1) What is the different between Chinese Restaurant Model and DP? None. CRP is a particular representation of DP. Depending on your problem you might want to use one representation over another (CRP, ...
  • 2,706
2 votes
Accepted

Dirichlet Process Hyperparameter Estimation with Sampling

Escobar and West give the standard Gibbs update for a gamma prior on the concentration parameter $\alpha$. Also note that the likelihood of the $\alpha$ given a partition $\mathscr P = \{b_1, b_2, \...
  • 8,102
2 votes

Polya Urn - formulas

I find the notion of $\alpha G_0(x)$ initial balls of color $x$ confusing and potentially misleading. I like another version of the urn better, similar to the one described on Wikipedia. Let's assume ...
  • 156
2 votes

Frequency distribution of Chinese Restaurant Process?

Your data {500,100,40,30,12} implies that 500+100+40+30+12 customers chose to sit at a new table, 100+40+30+12 customers chose to sit at a table that had exactly 1 customer at the time, 40+30+12 chose ...
2 votes
Accepted

Dirichlet Process with known mean

One approach would be to use the centered Dirichlet process described in the paper "Semiparametric Bayes hierarchical models with mean and variance constraints" http://ftp.stat.duke.edu/...
  • 86
2 votes

Posterior of parameter for Chinese Restaurant Process

Mike West wrote a method for this. The summary is that looked at in the right way, the posterior can be modelled as $p(α|k, n) ≈ G(a + k − 1, b + γ + log(n))$ Where $a,b$ are priors, $k$ is the ...
  • 4,041
2 votes

Nonparametric topic modeling: hierarchical dirichlet vs. Indian buffet?

Your intuition is not correct. IBP allows observations (customers) to share features (dishes), similarly HDP follows the metaphor of the Chinese Restaurant Franchise. In the CRF tables are shared ...
2 votes

Understanding Bayesian Histogram

Yes, I think your interpretation is correct. An important point is that the parameter vector $\pi$ in the model is assumed to follow a Dirichlet distribution. These parameters represents the ...
  • 41

Only top scored, non community-wiki answers of a minimum length are eligible