2
$\begingroup$

In this paper Dealing with Separation in Logistic Regression Models some various types of complete separation are discussed:

direction of the separation is positive if and only if $s_i = 1 \Rightarrow y_i = 1$ or $s_i = 0 \Rightarrow y_i = 0$

direction of the separation is negative if and only if $s_i = 0 \Rightarrow y_i = 1$ or $s_i = 1 \Rightarrow y_i = 0$

I'm wondering if this can be also called complete separation:

$s_i = 0 \Rightarrow y_i = 1$ AND $s_i = 1 \Rightarrow y_i = 1$

I call this one direction to distinct it from the other two.

Here I have the corresponding showcases to make it clear:

posivite direction:

          out
group    0  1
  ctrl  20  0
  treat  0 20

negative direction:

          out
group    0  1
  ctrl   0 20
  treat  20 0

one direction

          out
group    0  1
  ctrl   0 20
  treat  0 20

My questions are:

  • Can this be also called complete separation?
  • May I use the same tools (for example bayesglm from R) to analyze this kind of complete separation?
$\endgroup$

1 Answer 1

2
$\begingroup$

The problem with your question is that you treat logistic regression as a classifier, that is for you it outputs clases. But that is not logistic regression (LR), LR estimates ("outputs") probabilities, see for example Why isn't Logistic Regression called Logistic Classification? or Logistic regression - how good is my model? .

The paper you linked do not have a correct definition of complete separation, it says that only occurs if it is caused by one variable $s_i$. That is not correct, complete separation can well occur without any single variable causing it. So maybe you should find some better source of basic information on LR, maybe https://www.amazon.com/Regression-Modeling-Strategies-Applications-Statistics/dp/0387952322 (there are also many good posts on this site).

$\endgroup$
2
  • $\begingroup$ Thanks for reply and the very interesting links. I don't use logistic regression as classifier but as modelling tool to estimates odds ratio. I just wondered if it is correct to call separation if all outcomes of one predictor is 1. Thinking again about this special case I would just omit this predictor. $\endgroup$
    – giordano
    Oct 1, 2017 at 15:17
  • $\begingroup$ Why would you omit it? Does not make sense, that it separates shows that it is a very good predictor! $\endgroup$ Oct 1, 2017 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.