Tagged Questions

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

learn more… | top users | synonyms

0
votes
0answers
12 views

What “randomness induces dependencies” and “estimates … exhibit shrinkage” mean?

I am reading a paper named "Hierarchical Dirichlet Processes", whose first paragraph reads A recurring theme in statistics is the need to separate observations into groups, and yet allow the groups ...
0
votes
0answers
28 views

What is the intuition behind Dirichlet distribution?

looking for explanation on similar lines What is the intuition behind beta distribution?
0
votes
1answer
21 views

Can dummy variables be used to represent space in latent Dirichlet allocation?

Can dummy variables be used to represent space in latent Dirichlet allocation? I have a set of geocoded textual documents. I would like to use LDA to generate a topic model for the documents. ...
0
votes
0answers
14 views

Comparing support of Dirichlet

If (x1, x2, x3, x4) ~ Dirichlet(a1, a2, a3, a4), find P(x1 > x2)? I tried to use integration, but it didn't work out... Could anyone please provide any help? I greally apprecaite it!!
0
votes
1answer
36 views

What is a draw from a Dirichlet Process?

I know a draw from a K-D Dirichlet distribution is a probability vector of dimension K. But still it's not clear what is a draw from a Dirichlet Process. What I understood is that a draw from a ...
0
votes
1answer
47 views

PyMC3 Dirichlet Distribution

I am implementing a linear regression model in pymc3 where the unknown vector of weights is constrained to be a probability mass function, hence modelled as a Dirichlet distribution, as in the ...
0
votes
0answers
13 views

Recursively computing Dirichlet cdf.

I want to implement the Dirichlet CDF and I wonder if I can compute it recursively Following Frigyik et al., we know that if $Q \sim \text{Dir}(\alpha)$, then: $X_1 \sim \text{Beta}(\alpha_1, ...
1
vote
1answer
26 views

Draw a multinomial distribution from a Dirichlet distribution?

I have a very rough understanding of the Dirichlet distribution and already seen some visualizations of its pdf over the 2-simplex, i.e., $\alpha$ is a 3D vector. However, I still do not understand ...
0
votes
0answers
23 views

Simulation from Dirichlet distribution with WinBUGS

I have a question. Now I am learning WinBUGS, doing bayesian statistics. How, can I simulate a Dirichlet distribution (which is the posterior, for my model ...
0
votes
0answers
38 views

Dirichlet process mixture model in Python

My question is concerned with the practical issues of using this model. I've tried to use Dirichlet process mixture model from Scikit learn python package to find a number of clusters in my data (1D ...
1
vote
1answer
45 views

multivariate dirichlet for multiple imputation

I dealing with 3 covariates {x1, x2, x3} all three are discrete and contain missing data. ...
1
vote
1answer
39 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
3answers
47 views

Finite mixture models - Basic understanding

I have been reading lecture slides about Dirichlet Process. In page 22, there is a picture about the following finite mixture model. $$\phi _{k}\sim H\\ \pi \sim Dirichlet(\alpha /K,\dots,\alpha ...
0
votes
0answers
25 views

Expectation of a generalization of Dirichlet distribution

For the standard Dirichlet, the expectation of $X_i$ is $\alpha_i/\alpha_0$, where $\alpha_0 = \sum_i \alpha_i$ (http://en.wikipedia.org/wiki/Dirichlet_distribution). I am considering the following ...
2
votes
1answer
29 views

Dirichlet sample by normalising Gamma RVs

I know that if you sample $K$ random variables $(X_1, X_2, \dots, X_K)$ from Gamma distributions using shape parameters $(\alpha_1, \alpha_2, \dots \alpha_K)$ and a scale parameter $\theta = 1$ such ...
4
votes
0answers
44 views

Better prediction models with polling data?

I've been working on a project on measuring polls' accuracy in complex contexts (more than two candidates) where there are a small number of inaccurate polling data points. I thought it would be the ...
1
vote
0answers
29 views

Distribution of number of values less than cutoff within symmetric Dirichlet

Assume have a symmetric Dirichlet distribution with $a_1= \dots =a_k = a$ $ (X_1, \ldots, X_K)\sim\operatorname{Dir}(a) $ I am trying to determine the distribution of the number of values less than ...
2
votes
0answers
35 views

Distribution over a matrix of probabilities

I am interested in a problem where I need to infer the distribution of a matrix of probabilities, $\matrix{\Theta}$, where the rows and columns must sum to 1 and every entry lie in the range 0 and 1. ...
4
votes
1answer
72 views

Weighted sum of Dirichlets is Dirichlet

I am studying Sethuraman's paper on the Dirichlet process and having difficulty showing a lemma. He states: Let $\boldsymbol\gamma=(\gamma_1,...\gamma_k)$ and $\gamma=\sum_j \gamma_j$ and let ...
0
votes
1answer
21 views

What is an assignment matrix?

I'm trying to implement a topic model using a Latent Dirichlet allocation (LDA) algorithm. I'm using sentences as my dataset. What is Ck in the given instructions? The instructions are as follows: ...
1
vote
1answer
114 views

Unkown 6-sided dice. After 600 rolls frequency for all sides exactly equal. What is the chance, that rolling “6” with this dice has frequency > 1/6?

Although it is unknown dice, the symmetry of the evidence tells us, that we can treat the dice as fair, so the chance should be exactly 50%. But if we simulate it by hand, the result is less then ...
1
vote
0answers
79 views

difference between hierarchical dirichlet process and nested dirichlet process

There have some extensions to Dirichlet process. One is Hierarchical Dirichlet process, and another is Nested Dirichlet Process. What are the differences between these two? I once read the paper of ...
1
vote
1answer
128 views

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
4
votes
2answers
303 views

simulate dirichlet process in R

I am reading the paper of "Dirichlet Process Mixtures of Generalized Linear Models" authored by L. A. Hannah. If I would like to simulate the following model $$\mathcal{P}\sim ...
1
vote
2answers
162 views

stick breaking model of Dirichlet process

I have a question regarding sticking-breaking model of Dirichlet process, which is defined as follows: There are further statements that I am not clear that how to derive equation 1 from that ...
1
vote
0answers
39 views

multivariate distributions bounded on [0,1] where parameters can be solved for from known mode

I need some kind of multivariate distribution which has the following properties support is $x_1,$ $x_2$, ..., $x_K$ where all $x_i \in [0,1]$ Though not required it is true that elements of the ...
1
vote
0answers
30 views

What to do for Dirichlet distribution when elements of X vector may be zero

So I want to define a Dirichlet distribution over frequency vectors which are unit vectors whose elements represent the frequency with which different characters occur in a body of text. Trouble is ...
2
votes
0answers
42 views

Can Dirichlet and multinomial distributions be defined from their univariate distributions this way?

For Dirichlet distribution: $X := [x_1, \cdots, x_K] \in [0,1]^K$, $x_i \sim {\rm beta}(\alpha_i, \beta_i)$ and $\sum x_i = 1$. Can we say the distribution of $X$ is a Dirichlet distribution? If ...
2
votes
1answer
114 views

Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions: X1 X2 X3 X4 X5 0.10 0.20 0.50 0.12 0.08 0.07 0.24 0.55 0.04 0.10 ... ...
2
votes
1answer
205 views

How to calculate model error (like MSE) for a multivariate proportional response?

I have data where the response is multivariate and proportional (rows [observations] sum to 1). I am modelling this response using a Dirichlet regression via the DirichletReg R package where the ...
0
votes
0answers
40 views

Estimating a joint probability distribution with antimode

I am looking for a certain multivariate probability distribution function to fit to my data, but the usual multivariate normal distribution is unfit, since my data has a dent (antimode) instead of a ...
4
votes
3answers
333 views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...
5
votes
2answers
613 views

Laplace smoothing and Dirichlet prior

On the wikipedia article of Laplace smoothing (or additive smoothing), it is said that from a Bayesian point of view, this corresponds to the expected value of the posterior distribution, using a ...
2
votes
0answers
161 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
1
vote
1answer
93 views

How to draw samples from a Bayesian nonparamatric density estimation? [DPpackage]

I am trying to compute a Kernel Density from high dimensional data ($n > 2$). The underlying (generative) model is assumed unknown. The goal is to draw samples from this estimate, in a sense ...
4
votes
1answer
185 views

The meaning of representing the simplex as a triangle surface in Dirichlet distribution?

I'm reading from a book that introduces the Dirchilet distribution and then presented figures about it. But I was not really able to understand those figures. I attached the figure here at the bottom. ...
2
votes
1answer
139 views

What does it mean to sample a probability vector from a Dirichlet distribution?

I'm essentially learning about Latent Dirichlet Allocation. I'm watching a video here: http://videolectures.net/mlss09uk_blei_tm/ and stuck at minute 45 when he started to explain on sampling from the ...
3
votes
1answer
91 views

Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
2
votes
0answers
395 views

Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
0
votes
0answers
86 views

How to sample from this dirichlet distribution with an L1 prior?

I'd like to draw a sample from a distribution with p.d.f $$f(p,q,r,s) \sim \mathrm{e}^{-w(|p+q-r-s|+|p-q-r+s|+|p-q+r-s|)}p^aq^br^cs^d \mathbb{1}_{p+q+r+s=1}$$ $w > 0$ is a free parameter (which ...
1
vote
1answer
99 views

What is the Dirichlet equivalent of a Beta (1,1) distribution?

If parameter p ~ Beta(1,1), this would reflect we know nothing about parameter $p$. Generalizing to the multivariate case, how would the same be said about a vector $P$ of probability parameters ...
1
vote
0answers
40 views

What is an 'atom' and what are 'atomic weights'?

I have come across the following statement: A notable feature of the Hierarchical Dirichlet Process is that all Dirichlet Processes' $G_j$ share the same set of atoms and only the atom weights ...
1
vote
1answer
82 views

Latent Dirichlet Allocation - understanding the posterior

I have a problem understanding the posterior for computing LDA, stated in page 7 of Blei (2007). From my point of view, it's not exactly consistent with Bayes' theorem, as described here. Could anyone ...
1
vote
0answers
83 views

Dealing with 0 values when calculating the mle for a Dirichlet distribution

I have $N$ pmfs, and for each each $L$ samples. Each sample has a variable amount of $x$ values, but the $x$ values that they have can be matched. So for example: $$sample_1 \rightarrow\ x_1 = 0, x_2 ...
1
vote
1answer
280 views

How can I prevent Overflow for the Gamma function?

I have a Dirichlet distribution from which I'm maximizing $\alpha$ by calculating the log likelihood with the following equation $p\left(s|\alpha\right) = ...
0
votes
0answers
37 views

Being uniform in log of some parameter?

Alright, I am reading this book called "Bayesian Data Analysis" and it states this idea of being uniform in log of something which I don't have any clue what that might mean?? Example 1: Introduction ...
2
votes
1answer
155 views

Can log-likelihood function calculated value (M-step) be smaller after 1 EM-iteration?

I am applying a MAP log-likelihood approach in order to fit a Markov mixture model, where objective function to be maximized is given by the formula: $$ L(X|\Theta ...
1
vote
2answers
122 views

Implementing Latent Dirichlet Allocation - notation confusion

I am trying to implement LDA using the collapsed Gibbs sampler from http://www.uoguelph.ca/~wdarling/research/papers/TM.pdf the main algorithm is shown below I'm a bit confused about the notation ...
2
votes
1answer
646 views

Drawing from Dirichlet distribution

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. My question is, how can I draw a sample (a $K$-dimensional vector) ...
1
vote
2answers
268 views

Can someone give a simple guide of Dirichlet process clustering?

I devoted my last 3 days to understand Dirichlet process and Dirichlet process clustering but because of my lacking knowledge of stochastic process and measure theory (I did not take any course on ...