The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

learn more… | top users | synonyms

1
vote
1answer
36 views

stick breaking model of Dirichlet process

I have a question regarding sticking-breaking model of Dirichlet process, which is defined as follows: There are further statements that I am not clear that how to derive equation 1 from that ...
1
vote
0answers
30 views

multivariate distributions bounded on [0,1] where parameters can be solved for from known mode

I need some kind of multivariate distribution which has the following properties support is $x_1,$ $x_2$, ..., $x_K$ where all $x_i \in [0,1]$ Though not required it is true that elements of the ...
1
vote
0answers
22 views

What to do for Dirichlet distribution when elements of X vector may be zero

So I want to define a Dirichlet distribution over frequency vectors which are unit vectors whose elements represent the frequency with which different characters occur in a body of text. Trouble is ...
2
votes
0answers
32 views

Can Dirichlet and multinomial distributions be defined from their univariate distributions this way?

For Dirichlet distribution: $X := [x_1, \cdots, x_K] \in [0,1]^K$, $x_i \sim {\rm beta}(\alpha_i, \beta_i)$ and $\sum x_i = 1$. Can we say the distribution of $X$ is a Dirichlet distribution? If ...
1
vote
1answer
54 views

Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions: X1 X2 X3 X4 X5 0.10 0.20 0.50 0.12 0.08 0.07 0.24 0.55 0.04 0.10 ... ...
2
votes
1answer
73 views

How to calculate model error (like MSE) for a multivariate proportional response?

I have data where the response is multivariate and proportional (rows [observations] sum to 1). I am modelling this response using a Dirichlet regression via the DirichletReg R package where the ...
0
votes
0answers
22 views

Estimating a joint probability distribution with antimode

I am looking for a certain multivariate probability distribution function to fit to my data, but the usual multivariate normal distribution is unfit, since my data has a dent (antimode) instead of a ...
2
votes
1answer
105 views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...
5
votes
2answers
187 views

Laplace smoothing and Dirichlet prior

On the wikipedia article of Laplace smoothing (or additive smoothing), it is said that from a Bayesian point of view, this corresponds to the expected value of the posterior distribution, using a ...
1
vote
1answer
79 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
1
vote
1answer
67 views

How to draw samples from a Bayesian nonparamatric density estimation? [DPpackage]

I am trying to compute a Kernel Density from high dimensional data ($n > 2$). The underlying (generative) model is assumed unknown. The goal is to draw samples from this estimate, in a sense ...
4
votes
1answer
86 views

The meaning of representing the simplex as a triangle surface in Dirichlet distribution?

I'm reading from a book that introduces the Dirchilet distribution and then presented figures about it. But I was not really able to understand those figures. I attached the figure here at the bottom. ...
2
votes
1answer
69 views

What does it mean to sample a probability vector from a Dirichlet distribution?

I'm essentially learning about Latent Dirichlet Allocation. I'm watching a video here: http://videolectures.net/mlss09uk_blei_tm/ and stuck at minute 45 when he started to explain on sampling from the ...
1
vote
0answers
54 views

Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
2
votes
0answers
140 views

Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
0
votes
0answers
59 views

How to sample from this dirichlet distribution with an L1 prior?

I'd like to draw a sample from a distribution with p.d.f $$f(p,q,r,s) \sim \mathrm{e}^{-w(|p+q-r-s|+|p-q-r+s|+|p-q+r-s|)}p^aq^br^cs^d \mathbb{1}_{p+q+r+s=1}$$ $w > 0$ is a free parameter (which ...
1
vote
1answer
64 views

What is the Dirichlet equivalent of a Beta (1,1) distribution?

If parameter p ~ Beta(1,1), this would reflect we know nothing about parameter $p$. Generalizing to the multivariate case, how would the same be said about a vector $P$ of probability parameters ...
1
vote
0answers
38 views

What is an 'atom' and what are 'atomic weights'?

I have come across the following statement: A notable feature of the Hierarchical Dirichlet Process is that all Dirichlet Processes' $G_j$ share the same set of atoms and only the atom weights ...
1
vote
1answer
68 views

Latent Dirichlet Allocation - understanding the posterior

I have a problem understanding the posterior for computing LDA, stated in page 7 of Blei (2007). From my point of view, it's not exactly consistent with Bayes' theorem, as described here. Could anyone ...
1
vote
0answers
42 views

Dealing with 0 values when calculating the mle for a Dirichlet distribution

I have $N$ pmfs, and for each each $L$ samples. Each sample has a variable amount of $x$ values, but the $x$ values that they have can be matched. So for example: $$sample_1 \rightarrow\ x_1 = 0, x_2 ...
1
vote
1answer
130 views

How can I prevent Overflow for the Gamma function?

I have a Dirichlet distribution from which I'm maximizing $\alpha$ by calculating the log likelihood with the following equation $p\left(s|\alpha\right) = ...
0
votes
0answers
35 views

Being uniform in log of some parameter?

Alright, I am reading this book called "Bayesian Data Analysis" and it states this idea of being uniform in log of something which I don't have any clue what that might mean?? Example 1: Introduction ...
2
votes
1answer
106 views

Can log-likelihood function calculated value (M-step) be smaller after 1 EM-iteration?

I am applying a MAP log-likelihood approach in order to fit a Markov mixture model, where objective function to be maximized is given by the formula: $$ L(X|\Theta ...
0
votes
1answer
78 views

Implementing Latent Dirichlet Allocation - notation confusion

I am trying to implement LDA using the collapsed Gibbs sampler from http://www.uoguelph.ca/~wdarling/research/papers/TM.pdf the main algorithm is shown below I'm a bit confused about the notation ...
2
votes
1answer
197 views

Drawing from Dirichlet distribution

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. My question is, how can I draw a sample (a $K$-dimensional vector) ...
0
votes
2answers
111 views

Can someone give a simple guide of Dirichlet process clustering?

I devoted my last 3 days to understand Dirichlet process and Dirichlet process clustering but because of my lacking knowledge of stochastic process and measure theory (I did not take any course on ...
0
votes
0answers
61 views

can Dirichlet prior distribution be larger than 1?

This question is related to my quest of clustering the sequences using mixture Markov modeling. I have trouble understanding Dirichlet priors in the context of MAP-estimate (Mixture Markov Models). ...
1
vote
1answer
115 views

How to use prior probability in inferencing from HMM for activity recognition?

I am interested in modelling human activities using sensor data with HMMs and would like to incorporate prior knowledge during inference. The normal procedure is to model K different activities with K ...
0
votes
0answers
76 views

Dirichlet distribution

I have one question regarding Dirichlet distribution, I want to be sure if I understand it correctly: when I draw from K-dimensional Dirichlet, I get K-dimensional vector $X$ for which ...
0
votes
0answers
50 views

Negative binomial or Dirichlet regression - multiple DVs

I am researching a clinical study, where I am interested to see if high/low score on these 14 psychological constructs such as hostile attitudes (continuous, neg/pos values) can predict treatment ...
0
votes
0answers
94 views

How to compute similarity between two Dirichlet distributions?

I have a set of distributions and I would like to compute their similarity. Each distribution is a distribution over distributions of topics. If I have just a distribution over some topics, I could ...
0
votes
0answers
33 views

how do I show a convergence for the following?

Let $X_{i,j}$ be a set of random variables such that $j \le i$ and $i \in \{ 1, 2, \ldots \}$ It also holds that $$(X_{n,1},\ldots,X_{n,n}) \sim Dirichlet(c_0, c_1 X_{n-1,1}, \ldots, c_{n-1} ...
1
vote
1answer
141 views

What are typical values to use for alpha and beta in Latent Dirichlet Allocation?

Specifically in the case where I don't know anything about the documents I'm working with. I'm looking a specific number or number range.
1
vote
1answer
89 views

Categorical value “stuck” during sampling of my model

I'm having some troubles with the implementation in pyMC of my probabilistic model. Note: you can skip directly to the code section, if you're not interested on the use of the model. The model ...
2
votes
2answers
76 views

Dirichlet multinomial for adverse event data

I am trying to see which distribution will best fit the data I am working on. The dataset is as following: ...
7
votes
2answers
198 views

Implementation of Dirichlet cdf?

I need to compute the Dirichlet CDF, but I can only find implementations of the PDF. Do you guys know of any library (preferably in R) implementing it?
2
votes
0answers
123 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
1
vote
1answer
125 views

Derivation of the posterior over topics in LDA

When studying the latent Dirichlet allocation, I am not very clear about some procedures in their deriving equations. Please refer to the attached figure, how to understand those two steps, marked as ...
4
votes
1answer
132 views

Multivariate proportional data

I am looking for literature on what I call multivariate proportional data where a single observation is a vector of proportions that sum to 1. For example, each person weights their preferences for ...
4
votes
1answer
491 views

Bayesian estimation of Dirichlet distribution parameters

I want to estimate parameters of Dirichlet mixture models using Gibbs sampling and I have some questions about that: Is a mixture of Dirichlet distributions equivalent to a Dirichlet process? What ...
2
votes
1answer
105 views

How to find the MLE for two Dirichlet distributions with same mean

I have samples from two groups. Samples from each group follow a Dirichlet distribution. The two Dirichlet distributions have the same mean but different precision. How to find the MLE of mean and two ...
2
votes
2answers
155 views

kernel density estimation of a Dirichlet distribution

I have a Dirichlet distribution over the parameters of a multinomial, and I want to estimate its posterior density given some set of evidence. Due to some pecularities of my problem (e.g. presence of ...
1
vote
1answer
135 views

Latent Dirichlet analysis document comparison

I have been using Latent Dirichlet Analysis for a while but I am a bit confounded as to it's practical ability to compare two documents. It is of course ideal for classification when you want to see ...
0
votes
3answers
285 views

Bayes Estimation on Dirichlet distribution

I'm trying to get my head around the "hidden species" problem. It goes something like this. You visit a park and run into three species, 3 lions, 2 tigers and 1 bear. You are to determine what is your ...
1
vote
1answer
150 views

Nonparametric Topic Models

I am confused between what types of problems these three models capture, and their applications: Latent Dirichlet Allocation (LDA) Dirichlet Processes and Pitman-Yor processes Hierarchical Dirichlet ...
0
votes
1answer
349 views

Latent Dirichlet Allocation (LDA): What exactly is inferred?

I am working my way through LDA and I think I got they main idea of it. Please correct me if I am wrong. Given the Plate notation: The variables $\alpha$ and $\beta$ are Dirichlet distribution ...
8
votes
2answers
3k views

Why is the Dirichlet distribution the prior for the multinomial distribution?

In LDA topic model algorithm, I saw this assumption. But I don't know why chose Dirichlet distribution? I don't know if we can use Uniform distribution over Multinational as a pair?
5
votes
2answers
759 views

Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters

I will try to describe the problem at hand as general as possible. I am modeling observations as a categorical distribution with a parameter probability vector theta. Then, I assume the parameter ...
0
votes
1answer
71 views

Any actual applications based on “dirichlet distribution servers as a conjugate prior for multinomial distribution”

Conjugate prior is useful and beautiful in theory, for instance, Dirichlet distribution can serves as the conjugate prior for multinomial distribution. But I don't find any actual applications based ...
7
votes
1answer
168 views

Dirichlet posterior

I have a question about the Dirichlet posterior distribution. Given a multinomial likelihood function it's known that the posterior is $Dir({\alpha_i + N_i})$, where $N_i$ is the number of times we've ...