Let $X_i's$ be independent $i=1,2...n$ with $X_i\sim N(\mu+\alpha_i, \sigma^2)$ for each $i$. Let $\theta=(\alpha_1,...\alpha_p,\mu,\sigma)$ and $P_\theta$ be the joint pdf of the $X_i's$.

So, $P_\theta=(\frac{1}{\sqrt{2\pi}\sigma})^n e^{\frac{-1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu-\alpha_i)^2}$

From what I understand, a model $P_\theta$ is identifiable if $\theta_1=\theta_2$ implies that $P_{\theta_1}=P_{\theta_2}$ and $\theta_1\not=\theta_2$ implies that $P_{\theta_1}\not=P_{\theta_2}$

Proof by counterexample:

Let $\theta_1=(\alpha_1,...\alpha_p,\mu,\sigma)$ and $\theta_2=(\alpha_p,...\alpha_1,\mu,\sigma)$

$\theta_1\not=\theta_2 $ but $P_{\theta_1}=P_{\theta_2}$

So the model is unidentifiable?

  • 2
    $\begingroup$ It is not the case that $P_{\theta_1}=P_{\theta_2}$ (except if $\alpha_i=\alpha_{p+1-i}$ for $i=1, 2, \ldots, p$). $\endgroup$ – whuber Jul 18 '17 at 23:21
  • 1
    $\begingroup$ Consider some set of $\alpha$ and $\mu$ for $\theta_1$. Now in $\theta_2$ set its $\mu$, ($\mu_{(2)}$ say) to some particular value like $0$. Can you find a set of $\alpha_{(2),i}$ values that leave $E(W_i)$ unchanged? $\endgroup$ – Glen_b Jul 18 '17 at 23:31
  • $\begingroup$ @Glen_b Sorry, I edited the $W_i$ to $X_i$ to match what I used in the joint pdf. Ok, I think that if either $\mu$ or $\sigma$ changes, then the resulting distribution will also be different. What I was thinking of was that since the $W_i's$ are similarly normally distributed with the difference being their respective $\alpha_i's$, then interchanging the order of the $\alpha_i's$ in $\theta$ will give me different $\theta's$ but the same $P_{\theta}$. For example, for $n=2$, $E[(x_1-\mu-\alpha_1)^2+(x_2-\mu-\alpha_2)^2]=E[(x_1-\mu-\alpha_2)^2+(x_2-\mu-\alpha_1)^2]$ $\endgroup$ – user164144 Jul 18 '17 at 23:52
  • 1
    $\begingroup$ Don't interchange order. In my example consider $p=2$. Let $\mu_{(2)}=0$ Let $\alpha_{(2),i}=\mu+\alpha_i$. Leave $\sigma$ as is. $\endgroup$ – Glen_b Jul 19 '17 at 0:15
  • 1
    $\begingroup$ It might help to draw a picture of the distribution of $(X_1,X_2)$ in the case $p=2$ (consider contouring its density, for instance). Its mean will be at $(\mu+\alpha_1, \mu+\alpha_2)$. The distribution with the $\alpha_i$ switched will obviously have a different mean when $\alpha_1\ne\alpha_2$. That's all there is to it. $\endgroup$ – whuber Jul 19 '17 at 0:22

proof by counterexample Let $\theta=(\alpha_1,...\alpha_p, \mu.\sigma)$ s.t. $\mu=-\alpha_i$ for all $i$.

This means that $P_{\theta}=(\frac{1}{\sqrt{2\pi}\sigma})^n e^{\frac{-1}{2\sigma^2}\sum_{i=1}^n (x_i)^2}$

But this $\theta$ is not unique. We can have an infinite number of $\theta$ such that $\mu=-\alpha_i$ for all $i$. All of them would have the same $P_{\theta}$. Thus, the model is not identifiable.


One way to deal with a question like this is to "tweak" one of the parameters and see if you can derive how you need to modify the other parameters to compensate and get back to where you started from.

For example, set $\mu' = \mu - \Delta$, for some real $\Delta$. Then:

$$\begin{align} P_\theta&=\left(\frac{1}{\sqrt{2\pi}\sigma}\right)^n \exp\left({\frac{-1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu-\alpha_i)^2}\right)\\ &=\left(\frac{1}{\sqrt{2\pi}\sigma}\right)^n \exp\left({\frac{-1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu' + \Delta-\alpha_i)^2}\right)\\ &=\left(\frac{1}{\sqrt{2\pi}\sigma}\right)^n \exp\left({\frac{-1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu' - \alpha'_i)^2}\right)\\ \end{align}$$

where $\alpha'_i = \alpha_i + \Delta$ for each $i$. That is, the distribution is the same for any parameter vector $(\alpha'_1,...,\alpha'_n,\mu',\sigma)=(\alpha_1 + \Delta, ..., \alpha_n + \Delta, \mu - \Delta, \sigma)$, no matter the value of $\Delta$, so the model is not identifiable.

This also shows you exactly where the symmetry is: the parameter $\mu$ defines a global reference level that you measure the $\alpha_i$ from, but it has no real meaning and you could raise it or lower it without any impact by compensating with the $\alpha_i$. You can simply set $\Delta = \mu$ and measure them from zero instead to get an identified model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.