MCAR, MAR and EM

Question

I have a binary(1/0) classification task. I am trying to find $p(y = 1 | X)$ where $X$ is the vector of input variables and $y$ is the binary output label.

Suppose that for some records the output labels ($y$) are missing.

• Scenario 1: Labels are Missing Completely at Random (MCAR)

  While estimating the model, is there a difference between discarding records without labels vs. using Expectation Maximization to estimate missing labels?

• Scenario 2: Labels are Missing at Random (MAR) conditional on $X$

  While estimating the model, is there a difference between discarding records without labels vs. using Expectation Maximization to estimate missing labels?

• Scenario 3: Labels are Not Missing at Random (NMAR)

  In this case there is clearly a difference between discarding records without labels vs. using EM.

Which option is better? Is there an all-time winner or does it depend on missingness mechanism?

I am asking the difference in terms of model bias, and estimation efficiency.

Answer 1

Suppose $Y$ is a test result with pass (1) or fail (0) from the student. You have 100 students. If all of them gave you $Y$, you can estimate $P(Y=1)$. If 20 of them did not give you $Y$, and most of them due to shame of fail. Then you cannot use rest 80 $Y$ to estimate $P(Y=1)$. But If the reason has no relation of $Y$, for example, family emergency, sport activity,..., i.e., the probability of not reporting $Y$ is the same for $Y=0$ and $Y=1$, then you can use the 80 $Y$ to estimate the $P(Y=1)$.

Then you want to estimate $P(Y=1)$ separately for boys and girls. Among 100 students, 50 are girls and 50 are boys. But among 80 valid $Y$s, 45 are girls and 35 girls. Obviously it is not MCAR. But if same as before that the reason has no relation with $Y$, you can still use 45 $Y$s from girls to estimate $P(Y=1|girl)$ and use 35 $Y$s from girls to estimate $P(Y=1|boy)$. $s(a = 1|X=x)$ can be anything between 0-1 given it is not a function of $Y$.

Basically, missing of response variable has no relation with the value of response variable, you can do the analysis by drop the records with missed response variable. If missing is related to $Y$, then after dropping the records with missed response variable, the rest is not random sample anymore, so no statistical method is available.

For other approaches, at least you can use simulation to compare them with "discarding records without labels". I believe that "discarding records without labels" is the best.