0
$\begingroup$

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.2. Logistic Regression, the author says the following:

3.2. Logistic regression In logistic regression, we use maximum likelihood to find the parameter vector $\beta$ of the following model: $$P(y = 1 \mid x) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$ With sample selection bias we will instead fit: $$P(y = 1 \mid x, s = 1) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$ However, because we are assuming that $y$ is independent of $s$ given $x$ we have that $P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced. Asymptotically, as long as $P(s = 1 \mid x)$ is greater than zero for all $x$, the results on a selected sample approach the results on a random sample. In fact, this is true for any learner that models $P(y \mid x)$ directly. These are all local learners.

This part is unclear to me:

However, because we are assuming that $y$ is independent of $s$ given $x$ we have that $P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced.

What is meant by "the number of examples is reduced", and why is this the case?

$\endgroup$
0
$\begingroup$

Selection bias can occur when a portion of the sample is dropped from the analysis, e.g., because of missing data or loss to follow-up. The variable $s$ is an indicator of whether an individual unit remains in the sample, with $s=1$ indicating that the unit is in the sample. When the analysis is restricted to just those that remain, the number of units in the sample is reduced, because some units are dropped. That is all this sentence is saying. I presume the author used the word "example" because, in supervised learning conditions, one provides examples of classes to train the model. The point of this section is to illustrate that even if some units in the sample, i.e., some examples of the objects to classify, are dropped from the sample, the resulting model will not be affected by selection bias; the only change that will occur is that accompanying a smaller dataset (i.e., less precision).

$\endgroup$
2
  • $\begingroup$ Thanks for the answer. When you say "unit", are you referring to individual datum, such as a single image? $\endgroup$ – The Pointer Dec 20 '20 at 6:50
  • 1
    $\begingroup$ Yes, exactly. One row of a dataset. Of course it depends on the context. One can use logistic regression in many circumstances. $\endgroup$ – Noah Dec 20 '20 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.