4
$\begingroup$

I have been reading lecture slides about Dirichlet Process.

enter image description here

In page 22, there is a picture about the following finite mixture model.

$$\phi _{k}\sim H\\ \pi \sim Dirichlet(\alpha /K,\dots,\alpha /K)\\ Z_{i}\rvert\pi \sim Discrete(\pi )\\ x_{i}\rvert\phi _{z_{i}}\sim F(\cdot \vert \phi _{z_{i}})$$

I know the meaning of the following variables (Please correct me if I were wrong):

$N:$ Number of parameters/observations

$K:$ Number of mixture models

$\alpha:$ Dirichlet parameter

$\pi:$ Probability distribution on N variables

$x_{i}$ observed variables or data that we want to model

$H:$ Hyper-parameters, prior distribution about the K mixture models

$\phi _{k}:$ Parameters of the k-th model

$F(\cdot \vert \phi _{z_{i}})$:Individual mixture models

My questions are:

  1. What does $z_{i}$ mean and what is the relation with $\pi$ and $x_{i}$?

  2. What does $Z_{i}\rvert\pi \sim Discrete(\pi )$ mean?

  3. What does $x_{i}\rvert\phi _{z_{i}}\sim F(\cdot \vert \phi _{z_{i}})$ mean?

Thanks!

$\endgroup$
1
$\begingroup$

The Wikipedia site for Dirichlet Latent Allocation contains a brief nice description of the different parameters. $Discrete(\pi)$ stands for a multinomial distribution. It is meant any distribution that allows you to model the fact that each word of a document can be assigned to a topic from a finite set of topics.

$\endgroup$
2
$\begingroup$

At first a brief introduction: in the finite mixture models (HMM) you observe a time series $x_i, t =1,\dots, T$ and you suppose that at each time $t$ you are in a specific regime/state that are represented by the variable $z_i \in \mathbb{Z}^+$. The observations $x_i$ are distributed accordingly to a distribution $F(\cdot)$, but the parameter of the distribution depends on the state you are, so in if you are in state 1 ($z_i=1$) then $x_i \sim F(\phi_i) $ or equivalently $x_i \sim F(\phi_{z_i})$. Regarding the distribution of $Z_i$ i think there is a problem, in the standard HMM the variable assumed by $z_i$ depends on the value of $z_{i-1}$. Let $\boldsymbol{\pi}_i = (\pi_{i1}, \pi_{i2}...)$ be a vector of probability where $pi_{ij}$ mean the probability of be in the state $i$ and in the next time being in the state j, then the probability of $z_i \sim \boldsymbol{\pi}_{z_{i-1}}$

Now:

1) $z_i$ indicates the state on time $i$, it depends on the vector of probability $\pi_{z_{i-1}}$ and decide which parameter use in the distribution of $x_i$

2) is the distribution (multinomial) of $z_i$

3)is the distribution of $x_i$

$\endgroup$
1
$\begingroup$

In other word, associated to each $x_i$ is a random variable $z_i$. $z_i$ is interpreted as an indicator variable, indicating to which component of the mixture the observation $x_i$ belongs. $z_i=k$ imply observation $x_i$ is drawn from component $k$.

Note that $P(z_i=k)=\pi_k$.

$\endgroup$
  • $\begingroup$ Note that you need a dollar symbol to mark off your Latex expressions. I've put them in for you. $\endgroup$ – Silverfish Dec 12 '15 at 19:40
0
$\begingroup$

$Z_{i}$ is the random variable and ($z_{i}$ is a value it can take) representing the random variable producing $x_{i}$. Interpret $Z_{i}|\pi$ as $P(Z_{i}=z_{i}|\pi)$ and F(.|$\phi(z_{i})$ (this does not match up with your diagram) can be thought of as F(X|Z)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.