6
$\begingroup$

In Wikipedia the definition of a parametric model is the following:

A parametric model is a collection of distributions, each of which is indexed by a unique finite-dimensional parameter: $\mathcal{P}=\{\mathbb{P}_{\theta} : \theta \in \Theta\}$, where $\theta$ is a parameter and $\Theta \subseteq \mathbb{R}^d$ is the feasible region of parameters, which is a subset of d-dimensional.

I wonder what is $\Theta$ in the case of a hierarchical model. Is it composed of all the latent variables of the model or only the one at the top level? Does this include the hyper-parameters?

My concern is that this definition has an influence on the definition of model identifiability.

$\endgroup$
  • 1
    $\begingroup$ Is my question not interesting ? answerless ? or anything. I would be happy with any element. $\endgroup$ – peuhp Feb 28 '14 at 8:55
2
+25
$\begingroup$

I wonder what is $\Theta$ in the case of a hierarchical model. Is it composed of all the latent variables of the model or only the one at the top level? Does this include the hyper-parameters?

As far as I undestand, the definition you point out from Wikipedia outlines that a parametric model is collection $\mathcal{P}$ of some $\mathbb{P}_{\theta}$ distributions.

Each of these distributions has a finite dimensional vector of parameters $\theta$, and each of these $\theta$'s has a feasible region $\Theta$. For the model to be parametric, each $\theta$ must be finite-dimensional, that is, $\Theta \subseteq \mathbb{R}^d$. So, answering your question, $\Theta$ refers only to the "possible outcomes" of the parameters of one distribution.

For example:

Suppose you have a model with likelihood: $Y \sim Normal(\mu, \sigma)$, then the feasible set for $\theta = \begin{bmatrix} \mu \\ \sigma \end{bmatrix}$ of this Normal distribution is $\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$ (sorry for the abuse of notation here).

Now an example of an hierarchical model:

$$Y_1 \sim Normal(\mu_1, \sigma)$$ $$Y_2 \sim Normal(\mu_2, \sigma)$$ $$\mu_1 \sim Normal(\alpha, 10)$$ $$\mu_2 \sim Normal(\alpha, 10)$$

Following Wikipedia's definition, we would be for each distribution, respectively:

$$\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \\ (0; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$ $$\Theta = \begin{bmatrix} (-\infty; \infty) \end{bmatrix} \subseteq \mathbb{R}^2$$

(ok, this is also abuse of notation, you should have $\theta_i$ and $\Theta_i$ for each of the 4 distributions, but I hope you can get the idea.)

My concern is that this definition has an influence on the definition of model identifiability.

I don't see how this definition itself could impact the definition of model identifiability, let's also remember that Bayesian and frenquentist identifiability are different concepts and since you use Bayesian tag in your question, this discussion might be of interest.

$\endgroup$
  • $\begingroup$ Thanks for your answer. Yet from wikipedia: A statistical model is identifiable if $P_{\theta_1}=P_{\theta_2} \Rightarrow\quad \theta_1=\theta_2 \quad\ \text{for all } \theta_1,\theta_2\in\Theta$. (So it defintion relies complelty on what is $\Theta$). I do not see how your definition allow to discuss the identifiability of a HBM. Thanks in advance. $\endgroup$ – peuhp Mar 5 '14 at 8:08
1
$\begingroup$

The parameters have no concept of the hierarchy, nor should they. $\Theta$ is the space of possibilities. Consider the example:

$ Y_{ij} \sim \text{Bernoulli}(p_i),$

$p_i \sim \text{Beta}(\alpha, \beta),$

for $i = 1, \ldots, 10$ subjects and $j=1, \ldots 5$ binary outcomes within each subject.

In this case, $\Theta = (0, \infty) \times (0, \infty)$, the space of possibilities for $(\alpha, \beta)$. Once you have those, you can compute any probability.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.