6
$\begingroup$

In Pattern Recognition and Machine Learning Bishop writes about Bayes networks:

For practical applications of probabilistic models, it will typically be the highernumbered variables corresponding to terminal nodes of the graph that represent the observations, with lower-numbered nodes corresponding to latent variables. The primary role of the latent variables is to allow a complicated distribution over the observed variables to be represented in terms of a model constructed from simpler (typically exponential family) conditional distributions.

And after a few lines:

The hidden variables in a probabilistic model need not, however, have any explicit physical interpretation but may be introduced simply to allow a more complex joint distribution to be constructed from simpler components.

What do you think he means by this type of hidden variables (with no physical interpretation)?
What can be a simple example of this?
I thought about mixture of gaussians, but they don’t correspond to a situation where the variables we are interested are highernumbered.

$\endgroup$

2 Answers 2

3
$\begingroup$

The only reasonable answer to me seems that latent variables are the parameters of a distribution written as they were real variables, while they haven't any physical interpretation.
Bishop is always very precise and clear, I wonder why this time he didn't use the single word "parameters", that would have been enlightening.

$\endgroup$
3
$\begingroup$

First, note that observed variables and latent variables both have probability distributions, parameters are fixed.

A helpful example can be found in Koller and Friedman's PGM textbook (see below). Note that incorporating the latent variable H in the left-hand model reduces the parameter space of the overall graphical model. An I-equivalent graph can be drawn without the latent variable H (as in the right-hand model), but it may require many more parameters than a model that incorporates latent variables.

Choosing between the two is a modeling decision (that can come down to statistical vs. computational simplicity). H in the left-hand model need not have any physical representation, but it may be included or removed in the model for interpretability, or other requirements of the problem (e.g., sampling, inference). That is, there are often context-specific trade-offs that need to be made in determining the graph's structure.

from Koller and Friedman's PGM text

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.