Identifiability in generalized linear random effect model?

Question

Suppose I observe binary $Y_{ij}$ for $i = 1, ..., N$ and $j = 1, ..., J$ and I want to model $$\Pr(Y_{ij} = 1 \mid \lambda_{i}) = \Phi(\lambda_{ij}), \qquad [Y_{ij} \perp Y_{ij'} \mid \lambda_i]$$ where the vector $\lambda_{i} = (\lambda_{i1}, \ldots, \lambda_{iJ})$ is a multivariate-normal random effect, $\lambda_i \sim \mathcal N(\mu, \Sigma)$ and $\Phi(\cdot)$ is the probit link function (in general this could be any link function and we would run into approximately the same issues, but probit is easier to analyze with normal random effects). $\Sigma$ is not necessarily full-rank so this specification also covers random effects models of the form $\lambda_{ij} = x_j^T b_i$ where $\dim(b_i) \ll J$.

My question is: under what conditions on $(\mu, \Sigma)$ are they identified? My concern is due to the fact that it is well-known that the related multivariate probit model is unidentified in the absence of further restrictions (such as requiring $\Sigma$ to be a correlation matrix). The model as written is equivalent to a multivariate probit with mean $\mu$ and variance $\mathbf I + \Sigma$ so similar concerns should apply here.

For example, I feel confident heuristically that setting $$ \Sigma(\theta)_{jj'} = \theta_1 e^{-\theta_2|t_j - t_{j'}|} $$ for known $t$ leaves $\theta_1$ unidentified. On the other hand $\Sigma = X \Sigma_b X^T$ is a standard random effects covariance matrix and so is apparently identified as long as $\Sigma_b$ has small dimension. What can one say about (say) $$ \Sigma = \Sigma(\theta) + X\Sigma_b X^T $$ where $\Sigma(\theta)$ is defined as above?

Answer 1

The identifiability problem in the probit model occurs because each latent vector $\boldsymbol{\lambda}_i$ affects the observable outcome only through its sign, and so if the underlying parameters do not affect the distributions of the sign, they are not identifiable. To see this, we note that it is possible to rewrite your multivariate probit model in the alternative form:

$$Y_{ij} = \mathbb{I}(\lambda_{ij} \leqslant 0) \quad \quad \quad \boldsymbol{\lambda}_i \sim \text{IID N}(\boldsymbol{\mu},\boldsymbol{\Sigma}).$$

We can see from this model form that the parameters $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$ will affect the likelihood function only through their effect on the joint distribution of the $\text{sgn } \lambda_{ij}$ values. If we have two different parameter settings that give the same joint distribution for these sign values, then those different parameter settings are observationally equivalent. For example, if we take $\boldsymbol{\mu} = \alpha \boldsymbol{\mu}_0$ and $\boldsymbol{\Sigma} = \alpha^2 \boldsymbol{\Sigma}_0$ then the parameter $\alpha$ is not identifiable, since changing this parameter does not affect the joint distribution of the sign values of the latent variables.

---

Framing the model in terms of IID standard normal variables: Suppose we let $\boldsymbol{\Lambda} \equiv \boldsymbol{\Sigma}^{1/2}$ denote the principal square root of the covariance matrix $\boldsymbol{\Sigma}$ (which is also a symmetric non-negative definite matrix), so that $\boldsymbol{\Sigma} = \boldsymbol{\Lambda}^2$. We now create the standardised random vector:

$$\boldsymbol{\theta}_i \equiv \boldsymbol{\Lambda}^{-1} (\boldsymbol{\lambda}_i-\boldsymbol{\mu}) \sim \text{N}(\mathbf{0}, \mathbf{I}).$$

We can write the individual elements of our latent vector $\boldsymbol{\lambda}_i$ as:

$$\lambda_{ij} = [\boldsymbol{\mu}+\boldsymbol{\Lambda} \boldsymbol{\theta}_i]_j = \mu_j + \sum_{\ell=1}^J \Lambda_{j, \ell} \cdot \theta_{i,\ell}.$$

Using this standardised random vector, we can therefore rewrite your model as:

$$Y_{ij} = \mathbb{I} \Bigg( \sum_{\ell=1}^J \Lambda_{j, \ell} \cdot \theta_{i,\ell} \leqslant - \mu_j \Bigg) \quad \quad \quad \boldsymbol{\theta}_{i,\ell} \sim \text{IID N}(0, 1).$$

In this alternative framing of the model we have a matrix of IID standard normal values $\boldsymbol{\theta}_{i,\ell} \sim \text{IID N}(0, 1)$, and the parameters $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$ now appear inside the indicator function for the sign of the latent variable (the latter through the elements of its principal square root matrix). This form also allows you to see when changes in the parameters will be observationally equivalent.