How to tell the difference between a Latent Variable and a Non-Latent Variable?

Question

I am struggling to understand how statisticians define a Latent Variable.

Suppose the joint probability associated with a statistical model is:

$$ p(x, Z, \theta)$$

$x$ are the random variables, $Z$ are the latent variables and $\theta$ are the parameters of the model that need to be estimated.

Although I haven't fully understood this point yet, but usually we want to remove the Latent Variables $Z$ from the estimation process since we are not interested in them. So using the laws of probability, we can write:

$$L(\theta; X) = p(X | \theta) = \int p(X, Z | \theta) dZ$$

But this brings me to my question: In a given model, how do we know which variables are Latent and which variables are not Latent?

The most popular example that comes up in this regard is the Gaussian Mixture Model:

$$p(x, \pi_1, \pi_2, \mu_1, \sigma_1, \mu_2, \sigma_2 ) = \pi_1 \cdot N(x | \mu_1, \sigma_1) + \pi_2 \cdot N(x | \mu_2, \sigma_2)$$

As I understand, $\mu_1, \sigma_1, \mu_2, \sigma_2$ are the actual (i.e. Non-Latent) parameters we are interested in estimating (i.e. $\theta$) and $\pi_1, \pi_2$ are the Latent Variables (i.e. $Z$). But why exactly is it defined this way?

I can understand the logic that $\mu_1, \sigma_1, \mu_2, \sigma_2$ might be more important for the analysis compared to $\pi_1, \pi_2$.

But what I don't understand is why $\pi_1, \pi_2$ are considered as fundamentally unobservable/hidden/missing parameters whereas $\mu_1, \sigma_1, \mu_2, \sigma_2$ are not considered as unobservable/hidden/missing.

Technically, you don't observe any parameter - all you observe is data, and then using Maximum Likelihood Estimation, you create a formula to estimate parameters based on the data ... such that your parameter estimates were those that had the highest probability of generating the data you observed.

So to sum things up, in any estimation problem, all parameters being estimated are unobserved and hidden ... but why do we consider some of them as Latent and some of them as Non-Latent?

Previous posts consulted:

• Difference between latent and auxiliary variables
• When do you consider a variable is a latent variable?
• Why cannot MLE be implemented for Gaussian mixture model directly?
• Why is the marginal likelihood difficult/intractable to estimate?

Answer 1

I would say the fact that you integrate them out is what makes them latent variables; they no longer appear in the likelihood either as parameters or as data.

You could, in principle, regard the $z_i$ as parameters (and perhaps call them $\zeta_i$), and obtain a likelihood $p(X; \zeta,\theta)$ that would be written the same as $P(X|Z;\theta)$. The problem with doing that is that the maximum likelihood estimators are not consistent. It's a technically valid choice, but a bad one.

Or, if you were a Bayesian, you wouldn't make any technical distinction between latent variables and parameters. You would still need to make a basically similar decision about using a hierarchical prior over the $z_i$ (which is more like treating them as a latent variable) or using something like independent priors (which is more like treating them as parameters). Again, it would probably be better for estimation to use the hierarchical prior.