# Deriving the log-likelihood of an 'complicated' distribution

In 'Likelihood-based inference with singular information matrix' (Rotnitzky) an example is given as follows:

Suppose that $Y$ is normally distributed with mean $\beta$ and variance $\sigma^2$. There are available for study $n$ independent individuals but for each there is the possibility that the value of $Y$ cannot be observed. If the probability of not being able to observe $Y$ is assumed independent of the unobserved value the analysis proceeds with just the fully observed individuals. Suppose, however, that conditionally on $Y= y$ the probability of observing $y$ has the form $$\mathcal{P}_c(y; \alpha_0, \alpha_1) = \exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}$$ where $(\alpha_0, \alpha_1)$ are unknown parameters and $H(\cdot)$ is a known function assumed to have its first three derivatives at $\alpha_0$ non-zero. Interest may lie in small values of $\alpha_1$ and in particular in testing the null hypothesis $\alpha_1=0$.

We thus consider two random variables $(R,Y)$, where $R$ is binary, taking values 0 and 1. The value of $Y$ is observed if and only if $R=1$. The contribution of one individual to the log-likelihood is thus $$-r\log \sigma - r \frac{(y-\beta)^2}{(2\sigma)^2} + rH\left(\alpha_0+\alpha_1 \dfrac{y-\beta}{\sigma}\right) + (1-r)\log Q_c(\alpha_0,\alpha_1)$$

where $$Q_C(\alpha_0,\alpha_1) = E\{1-\mathcal{P}_c(Y;\alpha_0,\alpha_1)\}$$ is the marginal probability that $Y$ is not observed. For $n$ individuals the log-likelihood $L_n(\beta,\sigma, \alpha_0, \alpha_1)$ is the sum of $n$ such terms.

But I don't get how this log-likelihood was derived. I guess I would need something like $f_{Y,R}(y,r) = f_{Y|R}(y|r) \cdot f_R(r)$ but how does this work?

Also I know how $$f_R(r) = \left(\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right) \right\}\right)^r \cdot \left(1-\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}\right)^{1-r}$$ But then the log-likelihood would be $$r\cdot \left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}+(1-r) \cdot \log\left(1-\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}\right)$$

Why is $Q_c$ the expected value of $1-\mathcal{P}_c$?

• I upvoted to cancel the downvote. I'm not sure why it was downvoted in the first place, this seems like a perfectly legitimate question. You even showed your own attempt to construct a Likelihood function. Apr 12 '17 at 14:07

For one thing, $Y$ is independent of $R$ but not vice versa. So it makes more sense to decompose the distribution as $p(Y,R) = p(R \mid Y) p(Y).$ However, keep in mind that the Likelihood is defined to be the probability of the observed data, given the model parameters. If $R=0$ then $Y$ is not observed. Therefore the likelihood of an observation for which $R=0$ would be the marginal probability of $R=0$. That is, for a single observation for which $R=0$,
$$\begin{split} \mathcal{L}(Y,R=0) &= p(R=0) \\ &= \int p(R=0 \mid y) p(y) dy \\ &= \int \{ 1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\} p (y) dy \\ &= E\{1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\}. \end{split}$$ The key point here is that if $R=0$ then $Y$ is unobserved, so we're taking the probability of only the observed variable here, namely $p(R)$. In cases where $R=1$ we observe both $R$ and $Y$ so the likelihood contribution of this data point becomes $$\mathcal{L}(Y, R=1) = p(Y,R=1) = p(Y) p(R=1 \mid Y) = \mathcal{N}(Y \mid \beta, \sigma^2) \mathcal{P}_c(Y; \alpha_0, \alpha_1).$$ So to put it generally, the log likelihood for a single observation becomes $$\begin{split} \log \mathcal{L}(Y,R) &= R \log p(Y,R) + (1-R) p(R) \\ &= R \{ \log \mathcal{N}(Y \mid \beta, \sigma^2) + \log \mathcal{P}_c(Y; \alpha_0, \alpha_1) \} + (1-R)E\{1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\}. \end{split}$$
• "$Y$ is indepenednt of $R$ but not vice versa". This is wrong, pairwise stochastic independence is reciprocal. $R$ is independent of $Y$ because the event "$Y$ has/has not been observed" is not part of the support of the distrinbution of $Y$. It is only with respect to these events that $R$ is not independent. Apr 12 '17 at 14:40
• They are not pairwise independent. $R$ is defined to be the label of the event "$Y$ has/has not been observed", which is given by the distribution $p(R) = \mathcal{P}_c(Y)^R (1 - \mathcal{P}_c(Y)^{1-R}).$ You're right that the event of "$Y$ has/has not been observed" (i.e. $R$) is not in the support of $p(Y)$, but that tells us that $Y$ is independent of $R$, not vice versa. The converse statement is not true; $Y$ is very much part of the support of the distribution of the event of whether we observed $Y$ (i.e. the distribution of $R$). Apr 12 '17 at 15:01
• Well, whether $Y$ has been observed or not does not affect whether it has been realized or not. So the probabilities $P(Y\leq y \mid R=0)$ and $P(R=0\mid Y\leq y)$ are well defined. Applying the well known relation we then have $$P(Y\leq y \mid R=0) \cdot P(R=0) = P(R=0\mid Y\leq y)\cdot P(Y\leq y)$$ $Y$ is independent of $R$, so $P(Y\leq y \mid R=0) = P(Y\leq y)$, which leads us to $P(R=0) = P(R=0\mid Y\leq y)$, which tells us that $R$ is indpendent of $Y$ (after doing the same exercise for $R=1$ of course). Am I missing something here? Apr 12 '17 at 15:14
• I believe the mistake would be in saying that $P(Y \le y \mid R=1) = P(Y \le y)$. Suppose, hypothetically, that $P(R \mid Y) \equiv \mathcal{P}_c(Y) = \delta(Y-Z)$. That means that if we observed $Y$ then $Y$ is necessarily equal to $Z$. So even though the process in which $Y$ is drawn happens before (and thus is independent of) it is observed, whether or not we observed it gives us a hint as to what it was, so the conditional $p(Y \mid R) \ne P(Y)$. Of course that means that I was mistaken in my first sentence, so thanks for pointing that out. Apr 12 '17 at 15:36
• "...whether or not we observed it gives us a hint as to what it was". Well, Remember the classic prank: "Can you tel me what time it is? - Yes (then silence)", contrasted to "Can you tel me what time it is? - Yes, it is 12:15"... In our case we are asking "suppose that $Y$ has been observed. What are the probabilities that $Y$ took a value no greater than $y$?" There is no information about the value of $Y$ in the assertion that we have observed it. Apr 12 '17 at 15:51