# Can I run a logistic regression with a small sample size and perfect convergence?

I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are nominal with two levels, 0 or 1. The three independent variables, respectively, have 26, 17, and 9 "successes" (where the score is 1). The dependent variable has 44 "successes". However - the "successes" in the dependent variable ALWAYS overlap with one or more "successes" in the independent variable. A "success" in one independent variable means that the dependent variable was also a "success."

Given the small number of responses in my sample and the 1:1 overlap between "successes" in the independent and dependent variables, is it possible for me to run some variation of logistic regression on these variables to detect a relationship between the dependent and independent variables? Thanks in advance!

• It seems to me you just described a relationship. What do you seek beyond that? – whuber Jul 13 '18 at 0:43
• Fair point - the problem is that independent variable data were only collected for variables with a "success" in the dependent variable. What we want to know is which of the independent variables contributed most to those dependent variable successes across observations. – Andrew Jul 13 '18 at 4:30
• That is such a fundamentally important piece of information that it's crucial you include it in your post. Otherwise people will respond as if you had a random sample rather than such an extremely non-random one. – whuber Jul 13 '18 at 12:52

$$\hat{p} = \sigma(-10 + 20x_1 + 20x_2 + 20x_3)$$
where $\sigma$ is the sigmoid function. 10 is arbitrary and is chosen for concreteness: all that matter is the intercept is a large negative number and the coefficients are even larger positive numbers. When all three IVs are zero, this is $\sigma(-10) = \epsilon$ (where $\epsilon$ is a very small number) and if even one IV is a 1 then it will be at least $\sigma(10) =1 - \epsilon$. If two or more IVs are 1, it simply gets closer to 1. Because the decision threshold for for logistic regression is usually 0.5, every dependent class will be predicted with perfect accuracy.
Notice that this rule is of the form $x_1=1 \cup x_2=1 \cup x_3=1$, in other word it is an "OR" expression. Logistic regression can also learn "AND" expressions. Interesting enough, however, it cannot learn "XOR" rules unless you provide interaction terms in the design matrix. And in general it cannot learn arbitrary logical expressions. Compare this to say, a decision tree, which can. This limitation was one of the early motivations for exploring the multilayered perceptrons which eventually led to deep learning. But I digress.