In 'Likelihood-based inference with singular information matrix' (Rotnitzky) an example is given as follows:

Suppose that $Y$ is normally distributed with mean $\beta$ and variance $\sigma^2$. There are available for study $n$ independent individuals but for each there is the possibility that the value of $Y$ cannot be observed. If the probability of not being able to observe $Y$ is assumed independent of the unobserved value the analysis proceeds with just the fully observed individuals. Suppose, however, that conditionally on $Y= y$ the probability of observing $y$ has the form $$\mathcal{P}_c(y; \alpha_0, \alpha_1) = \exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}$$ where $(\alpha_0, \alpha_1)$ are unknown parameters and $H(\cdot)$ is a known function assumed to have its first three derivatives at $\alpha_0$ non-zero. Interest may lie in small values of $\alpha_1$ and in particular in testing the null hypothesis $\alpha_1=0$.

We thus consider two random variables $(R,Y)$, where $R$ is binary, taking values 0 and 1. The value of $Y$ is observed if and only if $R=1$. The contribution of one individual to the log-likelihood is thus $$-r\log \sigma - r \frac{(y-\beta)^2}{(2\sigma)^2} + rH\left(\alpha_0+\alpha_1 \dfrac{y-\beta}{\sigma}\right) + (1-r)\log Q_c(\alpha_0,\alpha_1)$$

where $$Q_C(\alpha_0,\alpha_1) = E\{1-\mathcal{P}_c(Y;\alpha_0,\alpha_1)\}$$ is the marginal probability that $Y$ is not observed. For $n$ individuals the log-likelihood $L_n(\beta,\sigma, \alpha_0, \alpha_1)$ is the sum of $n$ such terms.

But I don't get how this log-likelihood was derived. I guess I would need something like $f_{Y,R}(y,r) = f_{Y|R}(y|r) \cdot f_R(r)$ but how does this work?

Also I know how $$f_R(r) = \left(\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right) \right\}\right)^r \cdot \left(1-\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}\right)^{1-r}$$ But then the log-likelihood would be $$r\cdot \left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}+(1-r) \cdot \log\left(1-\exp\left\{H\left(\alpha_0+\alpha_1\dfrac{y-\beta}{\sigma}\right)\right\}\right) $$

Why is $Q_c$ the expected value of $1-\mathcal{P}_c$?

  • 1
    $\begingroup$ I upvoted to cancel the downvote. I'm not sure why it was downvoted in the first place, this seems like a perfectly legitimate question. You even showed your own attempt to construct a Likelihood function. $\endgroup$ Apr 12 '17 at 14:07

For one thing, $Y$ is independent of $R$ but not vice versa. So it makes more sense to decompose the distribution as $p(Y,R) = p(R \mid Y) p(Y).$ However, keep in mind that the Likelihood is defined to be the probability of the observed data, given the model parameters. If $R=0$ then $Y$ is not observed. Therefore the likelihood of an observation for which $R=0$ would be the marginal probability of $R=0$. That is, for a single observation for which $R=0$,

$$ \begin{split} \mathcal{L}(Y,R=0) &= p(R=0) \\ &= \int p(R=0 \mid y) p(y) dy \\ &= \int \{ 1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\} p (y) dy \\ &= E\{1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\}. \end{split} $$ The key point here is that if $R=0$ then $Y$ is unobserved, so we're taking the probability of only the observed variable here, namely $p(R)$. In cases where $R=1$ we observe both $R$ and $Y$ so the likelihood contribution of this data point becomes $$ \mathcal{L}(Y, R=1) = p(Y,R=1) = p(Y) p(R=1 \mid Y) = \mathcal{N}(Y \mid \beta, \sigma^2) \mathcal{P}_c(Y; \alpha_0, \alpha_1).$$ So to put it generally, the log likelihood for a single observation becomes $$ \begin{split} \log \mathcal{L}(Y,R) &= R \log p(Y,R) + (1-R) p(R) \\ &= R \{ \log \mathcal{N}(Y \mid \beta, \sigma^2) + \log \mathcal{P}_c(Y; \alpha_0, \alpha_1) \} + (1-R)E\{1 - \mathcal{P}_c(y; \alpha_0, \alpha_1)\}. \end{split} $$

EDIT: As Alecos Papadopoulos correctly pointed out, my first sentence is mistaken. What I should say is that it's more convenient to use my decomposition, but you can certainly decompose it in the other direction if you want. I don't think that this undermines the rest of my steps.

  • $\begingroup$ "$Y $ is indepenednt of $R$ but not vice versa". This is wrong, pairwise stochastic independence is reciprocal. $R$ is independent of $Y$ because the event "$Y$ has/has not been observed" is not part of the support of the distrinbution of $Y$. It is only with respect to these events that $R$ is not independent. $\endgroup$ Apr 12 '17 at 14:40
  • $\begingroup$ They are not pairwise independent. $R$ is defined to be the label of the event "$Y$ has/has not been observed", which is given by the distribution $p(R) = \mathcal{P}_c(Y)^R (1 - \mathcal{P}_c(Y)^{1-R}).$ You're right that the event of "$Y$ has/has not been observed" (i.e. $R$) is not in the support of $p(Y)$, but that tells us that $Y$ is independent of $R$, not vice versa. The converse statement is not true; $Y$ is very much part of the support of the distribution of the event of whether we observed $Y$ (i.e. the distribution of $R$). $\endgroup$ Apr 12 '17 at 15:01
  • $\begingroup$ Well, whether $Y$ has been observed or not does not affect whether it has been realized or not. So the probabilities $P(Y\leq y \mid R=0)$ and $P(R=0\mid Y\leq y)$ are well defined. Applying the well known relation we then have $$P(Y\leq y \mid R=0) \cdot P(R=0) = P(R=0\mid Y\leq y)\cdot P(Y\leq y)$$ $Y$ is independent of $R$, so $P(Y\leq y \mid R=0) = P(Y\leq y)$, which leads us to $P(R=0) = P(R=0\mid Y\leq y)$, which tells us that $R$ is indpendent of $Y$ (after doing the same exercise for $R=1$ of course). Am I missing something here? $\endgroup$ Apr 12 '17 at 15:14
  • $\begingroup$ I believe the mistake would be in saying that $P(Y \le y \mid R=1) = P(Y \le y)$. Suppose, hypothetically, that $P(R \mid Y) \equiv \mathcal{P}_c(Y) = \delta(Y-Z)$. That means that if we observed $Y$ then $Y$ is necessarily equal to $Z$. So even though the process in which $Y$ is drawn happens before (and thus is independent of) it is observed, whether or not we observed it gives us a hint as to what it was, so the conditional $p(Y \mid R) \ne P(Y)$. Of course that means that I was mistaken in my first sentence, so thanks for pointing that out. $\endgroup$ Apr 12 '17 at 15:36
  • $\begingroup$ "...whether or not we observed it gives us a hint as to what it was". Well, Remember the classic prank: "Can you tel me what time it is? - Yes (then silence)", contrasted to "Can you tel me what time it is? - Yes, it is 12:15"... In our case we are asking "suppose that $Y$ has been observed. What are the probabilities that $Y$ took a value no greater than $y$?" There is no information about the value of $Y$ in the assertion that we have observed it. $\endgroup$ Apr 12 '17 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.