4
$\begingroup$

I have a Chinese Restaurant Process with (unknown) concentration parameter $\theta$. After $n$ customers have been seated, I observe the number of non-empty tables and the number of people at each table. Now I'd like to estimate $\theta$.

How can I estimate $\theta$? e.g., how do I compute the maximum likelihood estimate for $\theta$, or how can I estimate the posterior distribution on $\theta$, given some reasonable/convenient prior?


Crudely, I suppose I could estimate the parameter $\theta$ from the number $t$ of non-empty tables and number $n$ of customers via $\theta \approx t / \log n$, based on a formula for the expected number of non-empty tables as a function of $\theta,n$. However, this doesn't take into account the full data I have, namely, the number of people sitting at each non-empty table.

$\endgroup$
1
$\begingroup$

Let $z_i = c$ signify that customer $i$ is seated at table $c$. Let $d_c$ denote the number of customers seated at table $c$. Let $m$ denote the number of tables with at least one customer. Then $\sum_{c=1}^m d_c = n$, since all the customers are seated somewhere. Let $z_{1:n} = (z_1, \ldots, z_n)$ and let $\theta$ denote the concentration parameter.

The distribution of $z_{1:n}$ given $\theta$ can be expressed as $$ p(z_{1:n}|\theta) = \frac{\theta^m \prod_{c=1}^m (d_c-1)!}{\Gamma(n+\theta)/\Gamma(\theta)} , $$ which can be obtained from $$ p(z_{1:n}|\theta) = p(z_1|\theta) \prod_{i=1}^{n-1} p(z_{i+1}|z_{1:i},\theta). $$ Therefore, the likelihood for $\theta$ is given by $$ p(z_{1:n}|\theta) \propto \frac{\theta^m\,\Gamma(\theta)}{\Gamma(n+\theta)} , $$ which depends only on the number of customers and the number of occupied tables.

The posterior distribution for $\theta$ then is $$ p(\theta|z_{1:n}) \propto p(z_{1:n}|\theta)\,p(\theta) . $$ I typically use a non-conjugate prior for $\theta$ that is proper but relatively ``open-minded:'' $$ p(\theta) = \textsf{Log-Logistic}(\theta|1,1) = \frac{1}{(1+\theta)^2} . $$ This distribution does not have a finite mean; its median equals 1. I use a Metropolis-Hastings scheme to sample from the posterior.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

Mike West wrote a method for this. The summary is that looked at in the right way, the posterior can be modelled as

$p(α|k, n) ≈ G(a + k − 1, b + γ + log(n))$

Where $a,b$ are priors, $k$ is the number of clusters, $\gamma$ Euler's constant and $n$ your number of observations. Note the approximation, he provides IIRC another method involving mixtures which has the exact conditional.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.