The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

learn more… | top users | synonyms

1
vote
1answer
40 views

multivariate dirichlet for multiple imputation

I dealing with 3 covariates {x1, x2, x3} all three are discrete and contain missing data. ...
1
vote
1answer
32 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
3answers
35 views

Finite mixture models - Basic understanding

I have been reading lecture slides about Dirichlet Process. In page 22, there is a picture about the following finite mixture model. $$\phi _{k}\sim H\\ \pi \sim Dirichlet(\alpha /K,\dots,\alpha ...
0
votes
0answers
19 views

Expectation of a generalization of Dirichlet distribution

For the standard Dirichlet, the expectation of $X_i$ is $\alpha_i/\alpha_0$, where $\alpha_0 = \sum_i \alpha_i$ (http://en.wikipedia.org/wiki/Dirichlet_distribution). I am considering the following ...
1
vote
1answer
25 views

Dirichlet sample by normalising Gamma RVs

I know that if you sample $K$ random variables $(X_1, X_2, \dots, X_K)$ from Gamma distributions using shape parameters $(\alpha_1, \alpha_2, \dots \alpha_K)$ and a scale parameter $\theta = 1$ such ...
4
votes
0answers
27 views

Better prediction models with polling data?

I've been working on a project on measuring polls' accuracy in complex contexts (more than two candidates) where there are a small number of inaccurate polling data points. I thought it would be the ...
1
vote
0answers
27 views

Distribution of number of values less than cutoff within symmetric Dirichlet

Assume have a symmetric Dirichlet distribution with $a_1= \dots =a_k = a$ $ (X_1, \ldots, X_K)\sim\operatorname{Dir}(a) $ I am trying to determine the distribution of the number of values less than ...
2
votes
0answers
32 views

Distribution over a matrix of probabilities

I am interested in a problem where I need to infer the distribution of a matrix of probabilities, $\matrix{\Theta}$, where the rows and columns must sum to 1 and every entry lie in the range 0 and 1. ...
4
votes
1answer
64 views

Weighted sum of Dirichlets is Dirichlet

I am studying Sethuraman's paper on the Dirichlet process and having difficulty showing a lemma. He states: Let $\boldsymbol\gamma=(\gamma_1,...\gamma_k)$ and $\gamma=\sum_j \gamma_j$ and let ...
0
votes
1answer
20 views

What is an assignment matrix?

I'm trying to implement a topic model using a Latent Dirichlet allocation (LDA) algorithm. I'm using sentences as my dataset. What is Ck in the given instructions? The instructions are as follows: ...
1
vote
1answer
88 views

Unkown 6-sided dice. After 600 rolls frequency for all sides exactly equal. What is the chance, that rolling “6” with this dice has frequency > 1/6?

Although it is unknown dice, the symmetry of the evidence tells us, that we can treat the dice as fair, so the chance should be exactly 50%. But if we simulate it by hand, the result is less then ...
1
vote
0answers
62 views

difference between hierarchical dirichlet process and nested dirichlet process

There have some extensions to Dirichlet process. One is Hierarchical Dirichlet process, and another is Nested Dirichlet Process. What are the differences between these two? I once read the paper of ...
1
vote
1answer
93 views

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
4
votes
2answers
180 views

simulate dirichlet process in R

I am reading the paper of "Dirichlet Process Mixtures of Generalized Linear Models" authored by L. A. Hannah. If I would like to simulate the following model $$\mathcal{P}\sim ...
1
vote
1answer
91 views

stick breaking model of Dirichlet process

I have a question regarding sticking-breaking model of Dirichlet process, which is defined as follows: There are further statements that I am not clear that how to derive equation 1 from that ...
1
vote
0answers
36 views

multivariate distributions bounded on [0,1] where parameters can be solved for from known mode

I need some kind of multivariate distribution which has the following properties support is $x_1,$ $x_2$, ..., $x_K$ where all $x_i \in [0,1]$ Though not required it is true that elements of the ...
1
vote
0answers
30 views

What to do for Dirichlet distribution when elements of X vector may be zero

So I want to define a Dirichlet distribution over frequency vectors which are unit vectors whose elements represent the frequency with which different characters occur in a body of text. Trouble is ...
2
votes
0answers
38 views

Can Dirichlet and multinomial distributions be defined from their univariate distributions this way?

For Dirichlet distribution: $X := [x_1, \cdots, x_K] \in [0,1]^K$, $x_i \sim {\rm beta}(\alpha_i, \beta_i)$ and $\sum x_i = 1$. Can we say the distribution of $X$ is a Dirichlet distribution? If ...
2
votes
1answer
94 views

Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions: X1 X2 X3 X4 X5 0.10 0.20 0.50 0.12 0.08 0.07 0.24 0.55 0.04 0.10 ... ...
2
votes
1answer
140 views

How to calculate model error (like MSE) for a multivariate proportional response?

I have data where the response is multivariate and proportional (rows [observations] sum to 1). I am modelling this response using a Dirichlet regression via the DirichletReg R package where the ...
0
votes
0answers
31 views

Estimating a joint probability distribution with antimode

I am looking for a certain multivariate probability distribution function to fit to my data, but the usual multivariate normal distribution is unfit, since my data has a dent (antimode) instead of a ...
4
votes
3answers
244 views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...
5
votes
2answers
435 views

Laplace smoothing and Dirichlet prior

On the wikipedia article of Laplace smoothing (or additive smoothing), it is said that from a Bayesian point of view, this corresponds to the expected value of the posterior distribution, using a ...
2
votes
1answer
139 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
1
vote
1answer
86 views

How to draw samples from a Bayesian nonparamatric density estimation? [DPpackage]

I am trying to compute a Kernel Density from high dimensional data ($n > 2$). The underlying (generative) model is assumed unknown. The goal is to draw samples from this estimate, in a sense ...
4
votes
1answer
143 views

The meaning of representing the simplex as a triangle surface in Dirichlet distribution?

I'm reading from a book that introduces the Dirchilet distribution and then presented figures about it. But I was not really able to understand those figures. I attached the figure here at the bottom. ...
2
votes
1answer
108 views

What does it mean to sample a probability vector from a Dirichlet distribution?

I'm essentially learning about Latent Dirichlet Allocation. I'm watching a video here: http://videolectures.net/mlss09uk_blei_tm/ and stuck at minute 45 when he started to explain on sampling from the ...
1
vote
0answers
68 views

Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
2
votes
0answers
288 views

Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
0
votes
0answers
74 views

How to sample from this dirichlet distribution with an L1 prior?

I'd like to draw a sample from a distribution with p.d.f $$f(p,q,r,s) \sim \mathrm{e}^{-w(|p+q-r-s|+|p-q-r+s|+|p-q+r-s|)}p^aq^br^cs^d \mathbb{1}_{p+q+r+s=1}$$ $w > 0$ is a free parameter (which ...
1
vote
1answer
87 views

What is the Dirichlet equivalent of a Beta (1,1) distribution?

If parameter p ~ Beta(1,1), this would reflect we know nothing about parameter $p$. Generalizing to the multivariate case, how would the same be said about a vector $P$ of probability parameters ...
1
vote
0answers
40 views

What is an 'atom' and what are 'atomic weights'?

I have come across the following statement: A notable feature of the Hierarchical Dirichlet Process is that all Dirichlet Processes' $G_j$ share the same set of atoms and only the atom weights ...
1
vote
1answer
76 views

Latent Dirichlet Allocation - understanding the posterior

I have a problem understanding the posterior for computing LDA, stated in page 7 of Blei (2007). From my point of view, it's not exactly consistent with Bayes' theorem, as described here. Could anyone ...
1
vote
0answers
62 views

Dealing with 0 values when calculating the mle for a Dirichlet distribution

I have $N$ pmfs, and for each each $L$ samples. Each sample has a variable amount of $x$ values, but the $x$ values that they have can be matched. So for example: $$sample_1 \rightarrow\ x_1 = 0, x_2 ...
1
vote
1answer
201 views

How can I prevent Overflow for the Gamma function?

I have a Dirichlet distribution from which I'm maximizing $\alpha$ by calculating the log likelihood with the following equation $p\left(s|\alpha\right) = ...
0
votes
0answers
37 views

Being uniform in log of some parameter?

Alright, I am reading this book called "Bayesian Data Analysis" and it states this idea of being uniform in log of something which I don't have any clue what that might mean?? Example 1: Introduction ...
2
votes
1answer
137 views

Can log-likelihood function calculated value (M-step) be smaller after 1 EM-iteration?

I am applying a MAP log-likelihood approach in order to fit a Markov mixture model, where objective function to be maximized is given by the formula: $$ L(X|\Theta ...
0
votes
1answer
99 views

Implementing Latent Dirichlet Allocation - notation confusion

I am trying to implement LDA using the collapsed Gibbs sampler from http://www.uoguelph.ca/~wdarling/research/papers/TM.pdf the main algorithm is shown below I'm a bit confused about the notation ...
2
votes
1answer
420 views

Drawing from Dirichlet distribution

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. My question is, how can I draw a sample (a $K$-dimensional vector) ...
1
vote
2answers
199 views

Can someone give a simple guide of Dirichlet process clustering?

I devoted my last 3 days to understand Dirichlet process and Dirichlet process clustering but because of my lacking knowledge of stochastic process and measure theory (I did not take any course on ...
0
votes
0answers
68 views

can Dirichlet prior distribution be larger than 1?

This question is related to my quest of clustering the sequences using mixture Markov modeling. I have trouble understanding Dirichlet priors in the context of MAP-estimate (Mixture Markov Models). ...
1
vote
1answer
140 views

How to use prior probability in inferencing from HMM for activity recognition?

I am interested in modelling human activities using sensor data with HMMs and would like to incorporate prior knowledge during inference. The normal procedure is to model K different activities with K ...
1
vote
1answer
184 views

What are typical values to use for alpha and beta in Latent Dirichlet Allocation?

Specifically in the case where I don't know anything about the documents I'm working with. I'm looking a specific number or number range.
1
vote
1answer
119 views

Categorical value “stuck” during sampling of my model

I'm having some troubles with the implementation in pyMC of my probabilistic model. Note: you can skip directly to the code section, if you're not interested on the use of the model. The model ...
2
votes
2answers
88 views

Dirichlet multinomial for adverse event data

I am trying to see which distribution will best fit the data I am working on. The dataset is as following: ...
7
votes
2answers
269 views

Implementation of Dirichlet cdf?

I need to compute the Dirichlet CDF, but I can only find implementations of the PDF. Do you guys know of any library (preferably in R) implementing it?
2
votes
0answers
170 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
1
vote
1answer
139 views

Derivation of the posterior over topics in LDA

When studying the latent Dirichlet allocation, I am not very clear about some procedures in their deriving equations. Please refer to the attached figure, how to understand those two steps, marked as ...
4
votes
1answer
165 views

Multivariate proportional data

I am looking for literature on what I call multivariate proportional data where a single observation is a vector of proportions that sum to 1. For example, each person weights their preferences for ...
4
votes
1answer
606 views

Bayesian estimation of Dirichlet distribution parameters

I want to estimate parameters of Dirichlet mixture models using Gibbs sampling and I have some questions about that: Is a mixture of Dirichlet distributions equivalent to a Dirichlet process? What ...