Identifiability Versus Convexity

Question

I'm a little unclear on the definitions of "identifiable" and "convex." Consider the case where $X_1, \ldots, X_n \overset{iid}{\sim} \text{Bernoulli}(p)$. Then our likelihood function is $L(p) = p^{\sum_i x_i}(1-p)^{n - \sum x_i}.$

1. This model is identifiable or one-to-one because $L(p_1) = L(p_2)$ implies $p_1 = p_2$. Indeed \begin{align*} p_1^{\sum_i x_i}(1-p_1)^{n - \sum x_i} & = p_2^{\sum_i x_i}(1-p_2)^{n - \sum x_i} \\ \iff \left[\frac{p_1}{1-p_1}\right]^{\sum x_i} \left[1-p_1 \right]^n&= \left[\frac{p_2}{1-p_2}\right]^{\sum x_i}\left[1-p_2 \right]^n \\ \iff \bar{x}\{\text{logit}(p_1) - \text{logit}(p_2)\} &+ \{\log(1-p_1) - \log(1-p_2)\} = 0 \\ \iff p_1 &= p_2 \end{align*} because the second to last line is linear in $\bar{x}$.

2. But on the other hand it (the negative of it) is convex. If you plot it in $p$, it looks like an upside down bowl.

Does the difference have something to do with considering the data as random, as in the first case when we think of for all the $\bar{x}$s that are possible, and fixing the data, as in the second case? It's not clicking for me right now.

Edit: I thought of this question after I read this. I always thought identfiability meant uniqueness of the MLE, so I'm thinking maybe there is a connection.

Answer 1

Nonidentifiability does imply that the likelihood is possibly not unimodal, which is related. Taking the contrapositive of this, that means that unimodality implies identifiability.

Take any $\theta_1 \neq \theta_2$ for which the likelihood measures are the same. There must be at least one pair by the defintion of nonidentifiability. Then in particular they evaluate to the same scalar for any common dataset. If either of these parameters is the global maximum, then they both are.

To go any further than that you would probably need to look at the type of nonidentifiability. For example, aliasing would guarantee that you could find another $\theta_2$ for any $\theta_1$. Take for instance the model $Y \sim \text{Normal}(0,\theta^2)$ with $\theta \in \mathbb{R}$. Then for any $\theta$, and in particular the one that maximizes the likelihood for your specific data set, you can get the same likelihood by plugging in $-\theta$.