1
$\begingroup$

I have been trying to understand EM and I am having a hard time understanding what a latent variable is. In particular, I am having issues in identifying whether in a particular model that I am using, a particular variable is a latent variable or not.

So, basically I observe two images $x$ and $y$ and I am trying to estimate a transformation $t$ between them which is parameterised by a set of parameters $w$. So the model is: $$ y = t(x, w) + e $$ So, $y$ is a transformed version of $x$ according to some transformation parameters $w$. Graphically, I can represent it as follows.

enter image description here

Please ignore the prior distributions on $w$ and $\phi$ and all the hyper parameters.

So, my understanding is that $x$ and $y$ are two random variables but they are not latent as we actually observe and measure particular instances of them (hence they are shaded in the PGM diagram).

However, the parameters of the transformation $w$ can be treated as a random variable and since we do not observe it but rather want to infer it, they are the latent variables in my model.

Is my understanding/reasoning correct?

$\endgroup$
1
$\begingroup$

Hm... if you have data on $x$ and $y$ and assume some distribution of the error given $x$ then I would just maximize log likelikhood. I don't think there is any latent variable then.

But maybe you are not given (or don't want to assume) anything about the parameters of $P(e|x)$. If that is the case then it seems to me that parameters of that distributions would be latent variables.

If you do not want to assume anything about $P(e|x)$ then I don't think you can estimate w's. This is because whatever w's you choose you will always find a distribution $P(e|x)$ that will make the model fit perfectly.

$\endgroup$
  • $\begingroup$ Thanks for the answer! I am assuming that the noise model is a zero mean IID, so the noise variance is being inferred as well (denoted by the variable $\sigma$). My confusion is why is $w$ not a latent variable as you say when I assume zero mean IID noise. $w$ is still being inferred and is not measured/observed. $\endgroup$ – Luca Aug 11 '14 at 22:15
  • 1
    $\begingroup$ From what you are describing it seems that w is a constant in your model which value you are trying to figure out. $\endgroup$ – marcin_j Aug 11 '14 at 22:58
  • 1
    $\begingroup$ But it seems to me that the problem your model is trying to solve is actually different that you described. If w was a variable which we do not observe and we are trying to figure out what the parameters of w's distribution are then w would be a latent variable. Parameters you would be after would be the one that describe its distribution. I'm not sure if that helps, but hopefully gives some food for thought:) $\endgroup$ – marcin_j Aug 11 '14 at 23:00
  • $\begingroup$ There is a distribution attached to $w$ and it may have some complex form a-posteriori. So in a sense it is not a constant and can take a range of values with a certain probability. $\endgroup$ – Luca Aug 11 '14 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.