# Identifiability Versus Convexity

I'm a little unclear on the definitions of "identifiable" and "convex." Consider the case where $X_1, \ldots, X_n \overset{iid}{\sim} \text{Bernoulli}(p)$. Then our likelihood function is $L(p) = p^{\sum_i x_i}(1-p)^{n - \sum x_i}.$

1. This model is identifiable or one-to-one because $L(p_1) = L(p_2)$ implies $p_1 = p_2$. Indeed \begin{align*} p_1^{\sum_i x_i}(1-p_1)^{n - \sum x_i} & = p_2^{\sum_i x_i}(1-p_2)^{n - \sum x_i} \\ \iff \left[\frac{p_1}{1-p_1}\right]^{\sum x_i} \left[1-p_1 \right]^n&= \left[\frac{p_2}{1-p_2}\right]^{\sum x_i}\left[1-p_2 \right]^n \\ \iff \bar{x}\{\text{logit}(p_1) - \text{logit}(p_2)\} &+ \{\log(1-p_1) - \log(1-p_2)\} = 0 \\ \iff p_1 &= p_2 \end{align*} because the second to last line is linear in $\bar{x}$.

1. But on the other hand it (the negative of it) is convex. If you plot it in $p$, it looks like an upside down bowl.

Does the difference have something to do with considering the data as random, as in the first case when we think of for all the $\bar{x}$s that are possible, and fixing the data, as in the second case? It's not clicking for me right now.

Edit: I thought of this question after I read this. I always thought identfiability meant uniqueness of the MLE, so I'm thinking maybe there is a connection.

• Because there's no conceptual or mathematical connection between these two definitions, it's a little hard to figure out what you are really asking for. The function $L(p_1,p_2)=-(p_1-p_2)^2$, for $(p_1,p_2)\in\mathbb{R}^2$, is convex but not one-to-one: is this the sort of thing you are looking for? – whuber Jan 1 '17 at 20:08
• Are you asking, then, what "identifiability" really means? The issue of injectivity may be the real source of confusion, because obviously any continuous function with even a local maximum in the interior of its domain is not injective! – whuber Jan 1 '17 at 20:16
• @whuber right, yes. I don't really get the injective definition for identifiability. I don't see why lacking that complicates inference. Who cares if two values yield the same likelihood evaluation? As long as the argmax is unique, what do we care? – Taylor Jan 1 '17 at 20:19
• It can help to ponder the simplest possible cases of non-identifiability. Let $X$ have a unit-variance Normal distribution of unknown mean $\alpha-\beta$ where $\alpha$ and $\beta$ are real parameters to be estimated. The likelihood for $X=x$ is maximized when $\alpha-\beta=x$: it's an entire line of values. I believe one of the salient properties of this problematic situation is that the likelihood never has a unique maximum, no matter what value $x$ might be. – whuber Jan 1 '17 at 20:50
• So when we're talking about identifiability and we say two likelihoods are equal, we're thinking of equality of measures then I guess. As in they yield all the same probabilities, no matter the data. Makes sense. I haven't quite figured out why that works with uniqueness of MLEs though. Will keep thinking on it – Taylor Jan 2 '17 at 4:28

Take any $$\theta_1 \neq \theta_2$$ for which the likelihood measures are the same. There must be at least one pair by the defintion of nonidentifiability. Then in particular they evaluate to the same scalar for any common dataset. If either of these parameters is the global maximum, then they both are.
To go any further than that you would probably need to look at the type of nonidentifiability. For example, aliasing would guarantee that you could find another $$\theta_2$$ for any $$\theta_1$$. Take for instance the model $$Y \sim \text{Normal}(0,\theta^2)$$ with $$\theta \in \mathbb{R}$$. Then for any $$\theta$$, and in particular the one that maximizes the likelihood for your specific data set, you can get the same likelihood by plugging in $$-\theta$$.