# How to tell the difference between a Latent Variable and a Non-Latent Variable?

I am struggling to understand how statisticians define a Latent Variable.

Suppose the joint probability associated with a statistical model is:

$$p(x, Z, \theta)$$

$$x$$ are the random variables, $$Z$$ are the latent variables and $$\theta$$ are the parameters of the model that need to be estimated.

Although I haven't fully understood this point yet, but usually we want to remove the Latent Variables $$Z$$ from the estimation process since we are not interested in them. So using the laws of probability, we can write:

$$L(\theta; X) = p(X | \theta) = \int p(X, Z | \theta) dZ$$

But this brings me to my question: In a given model, how do we know which variables are Latent and which variables are not Latent?

The most popular example that comes up in this regard is the Gaussian Mixture Model:

$$p(x, \pi_1, \pi_2, \mu_1, \sigma_1, \mu_2, \sigma_2 ) = \pi_1 \cdot N(x | \mu_1, \sigma_1) + \pi_2 \cdot N(x | \mu_2, \sigma_2)$$

As I understand, $$\mu_1, \sigma_1, \mu_2, \sigma_2$$ are the actual (i.e. Non-Latent) parameters we are interested in estimating (i.e. $$\theta$$) and $$\pi_1, \pi_2$$ are the Latent Variables (i.e. $$Z$$). But why exactly is it defined this way?

I can understand the logic that $$\mu_1, \sigma_1, \mu_2, \sigma_2$$ might be more important for the analysis compared to $$\pi_1, \pi_2$$.

But what I don't understand is why $$\pi_1, \pi_2$$ are considered as fundamentally unobservable/hidden/missing parameters whereas $$\mu_1, \sigma_1, \mu_2, \sigma_2$$ are not considered as unobservable/hidden/missing.

Technically, you don't observe any parameter - all you observe is data, and then using Maximum Likelihood Estimation, you create a formula to estimate parameters based on the data ... such that your parameter estimates were those that had the highest probability of generating the data you observed.

So to sum things up, in any estimation problem, all parameters being estimated are unobserved and hidden ... but why do we consider some of them as Latent and some of them as Non-Latent?

Previous posts consulted:

You could, in principle, regard the $$z_i$$ as parameters (and perhaps call them $$\zeta_i$$), and obtain a likelihood $$p(X; \zeta,\theta)$$ that would be written the same as $$P(X|Z;\theta)$$. The problem with doing that is that the maximum likelihood estimators are not consistent. It's a technically valid choice, but a bad one.
Or, if you were a Bayesian, you wouldn't make any technical distinction between latent variables and parameters. You would still need to make a basically similar decision about using a hierarchical prior over the $$z_i$$ (which is more like treating them as a latent variable) or using something like independent priors (which is more like treating them as parameters). Again, it would probably be better for estimation to use the hierarchical prior.
• If you take a Gaussian mixture model and treat the cluster memberships as parameters you end up with $k$-means clustering; the difference between them is discussed here: stats.stackexchange.com/questions/489459/… and probably other questions as well Nov 7, 2023 at 2:06