Definition of statistical model in case of hierarchical model

Question

In Wikipedia the definition of a parametric model is the following:

A parametric model is a collection of distributions, each of which is indexed by a unique finite-dimensional parameter: $\mathcal{P}=\{\mathbb{P}_{\theta} : \theta \in \Theta\}$, where $\theta$ is a parameter and $\Theta \subseteq \mathbb{R}^d$ is the feasible region of parameters, which is a subset of d-dimensional.

I wonder what is $\Theta$ in the case of a hierarchical model. Is it composed of all the latent variables of the model or only the one at the top level? Does this include the hyper-parameters?

My concern is that this definition has an influence on the definition of model identifiability.

Answer 1

I wonder what is $\Theta$ in the case of a hierarchical model. Is it composed of all the latent variables of the model or only the one at the top level? Does this include the hyper-parameters?

As far as I undestand, the definition you point out from Wikipedia outlines that a parametric model is collection $\mathcal{P}$ of some $\mathbb{P}_{\theta}$ distributions.

Each of these distributions has a finite dimensional vector of parameters $\theta$, and each of these $\theta$'s has a feasible region $\Theta$. For the model to be parametric, each $\theta$ must be finite-dimensional, that is, $\Theta \subseteq \mathbb{R}^d$. So, answering your question, $\Theta$ refers only to the "possible outcomes" of the parameters of one distribution.

For example:

Suppose you have a model with likelihood: $Y \sim Normal(\mu, \sigma)$, then the feasible set for $\theta = \begin{bmatrix} \mu \\ \sigma \end{bmatrix}$ of this Normal distribution is $\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$ (sorry for the abuse of notation here).

Now an example of an hierarchical model:

$$Y_1 \sim Normal(\mu_1, \sigma)$$ $$Y_2 \sim Normal(\mu_2, \sigma)$$ $$\mu_1 \sim Normal(\alpha, 10)$$ $$\mu_2 \sim Normal(\alpha, 10)$$

Following Wikipedia's definition, we would be for each distribution, respectively:

$$\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$

(ok, this is also abuse of notation, you should have $\theta_i$ and $\Theta_i$ for each of the 4 distributions, but I hope you can get the idea.)

My concern is that this definition has an influence on the definition of model identifiability.

I don't see how this definition itself could impact the definition of model identifiability, let's also remember that Bayesian and frenquentist identifiability are different concepts and since you use Bayesian tag in your question, this discussion might be of interest.

Answer 2

The parameters have no concept of the hierarchy, nor should they. $\Theta$ is the space of possibilities. Consider the example:

$ Y_{ij} \sim \text{Bernoulli}(p_i),$

$p_i \sim \text{Beta}(\alpha, \beta),$

for $i = 1, \ldots, 10$ subjects and $j=1, \ldots 5$ binary outcomes within each subject.

In this case, $\Theta = (0, \infty) \times (0, \infty)$, the space of possibilities for $(\alpha, \beta)$. Once you have those, you can compute any probability.