1
$\begingroup$

I'm a programmer and currently trying to apply the Latent Dirichlet Allocation algorithm by Blei et al. on a text mining problem. I am using a library called gensim for this, which takes, among others, a vector of alpha values as parameters - the same $\alpha$ values that determine the Dirichlet distribution.

So, if we specifiy $k$, we have a vector of $k$ $\alpha$-values. For $k = 3$ we may choose our $\alpha$ values like this: https://en.wikipedia.org/wiki/Dirichlet_distribution#mediaviewer/File:Dirichlet_distributions.png.

If I understood it correctly, a "flat"/neutral Dirichlet distribution can be achieved by choosing $\{a_k\}=1$. First question: Is that correct? I've read this in chapter 2.2.1 ("The Dirichlet Distribution") of "Pattern Recognition and Machine Learning" by Christopher M. Bishop, if that's of any interest.

The problem is that the library I use takes the alpha values only in normalized form, which means that all $\alpha$ values have to sum up to 1. Therefore my second question is: How do I normalize $\alpha$ values? I've found a normalization constant on https://en.wikipedia.org/wiki/Dirichlet_distribution#Probability_density_function, but it seems to apply to the probability density function only; otherwise it doesn't quite work out (the $\alpha$ values don't sum up to 1).

I am asking the second question because if, for example, $\alpha = \{1, 1, 1\}$ generates a "flat" Dirichlet distribution and $\alpha = \{5, 5, 5\}$ does not, I can't see how to normalize these values without the former having the same normalized values as the latter (e.g. $\alpha_{normalized} = \{\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\}$), which would mean that they'd generate the same distribution.

Again, I'm not a mathematician, and I'm pretty sure this probably seems like a pretty stupid question to you. Don't be too harsh, please :-)

I'd appreciate any help!

Best Regards & thanks for your time, MG

Edit: Unfortunately I don't seem to be allowed to neither comment nor accept answers since I posted this question without an account, so in response to...

  • @tristan: Yeah, that's what I thought. Something like a concentration parameter would be necessary.
  • @whuber: Well, turns out I just misunderstood the library's documentation. The library allows for normalized parameter, but also accepts non-normalized data (e.g. {1, 1, 1}).

So, sorry for the confusion, guys. And thanks for your answers!

$\endgroup$
  • $\begingroup$ It has been suggested that stats.SE is a better venue for this question, and I am inclined to agree. Migrating now... $\endgroup$ – Todd Trimble Oct 20 '14 at 2:04
  • $\begingroup$ Are you perhaps confusing normalization of the variates with normalization of the parameters? Any library that insists the parameters sum to unity cannot be a full implementation of the Dirichlet distribution (and it would be strange for the library to be so limited). $\endgroup$ – whuber Oct 20 '14 at 16:00
  • $\begingroup$ This is a great question and I think you worded it well. I wish it had more discussion answers. $\endgroup$ – O.rka Nov 8 '16 at 18:09
1
$\begingroup$

In answer to your first question, yes: if $ a_i = 1$ for all $i$ you get the uniform distribution. For second question I suspect you need to find a concentration parameter (call it $r$ for sake of argument), so that you would specify $ r=3$ and $ a_1=a_2=a_3=1/3$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy