# What is the connection between lift and logistic regression?

I have noticed that there is an interesting connection between two (apparently different) measures. I am under a market basket analysis framework (aka frequent itemset mining, both are common names) , where all the variables are hot-encoded. So our predicted variable Y and our predictor variable X are both dummy variables which represents the existance or non-existance of a product in a transaction.

The first measure 'lift' comes from apriori algorithm.

it is defined as $$\frac{confidence(A \rightarrow B)}{support(B)} = \frac{P(B|A)}{P(B)}$$ which is basically $$\frac{P(A \cap B)}{P(A)P(B)}$$

The second measure comes from a univariate logistic regression.

$$logit(Y) = a + bX + e_i$$

My doubt is in the logistic regression part. a is the intercept and b represents the difference in log odds.

Both measures are symmetric, when regressing Y on X or X on Y (or when using apriori) I still get the same lift and b. But they have different scales. How is this related? Is there a form I can translate from b values in a logistic regression to a lift equivalent?

Any hints or bibliography where I can find more information is highly appreciated.

Edit:

Here I attach a small sample. These represent some rules extracted by the apriori algorithm. Column Antecedent represents the product A, Column Consequent represents product B. Lift and logistic_coefficients are our variables of interest in this case, and I am trying to figure out how are they related and if we can map from one to the other one.

(In case someone is interested, here's the link for other results)

The lift is a combination of the difference in the odds (related to the coefficient $$b$$), and the probability for the cases $$x=0$$ and $$x=1$$.

We can write $$P(Y) = logistic(a) P(\lnot X) + logistic(a+b) P(X)$$ and $$P(Y|X) = logistic(a+b)$$ making

$$\begin{array}{} \frac{P(Y|X)}{P(Y)} & =& \frac{1}{P(X)+\frac{logistic(a)}{logistic(a+b)} P(\lnot X)} \\ &=& \frac{1}{1+\frac{e^{-a}(e^{-b}-1)}{e^{-a}+1}P(\lnot X)} \end{array}$$

An example below gives the same logistic regression but different lift.

### Table 1

$$\begin{array}{c|cccc} & Y =0& Y=1 \\ \hline X=0 & 0.1 & 0.1 \\ X = 1 & 0.2 & 0.6 \end{array}$$

The odds are

$$\frac{P(Y=1|X=0)}{P(Y=0|X=0)} = \frac{0.1}{0.1}= 1 \\ \frac{P(Y=1|X=1)}{P(Y=0|X=1)} = \frac{0.6}{0.2} = 3$$

The lift is

$$\frac{P(Y=1|X=1)}{P(Y=1)} = \frac{0.75}{0.7} \approx 1.071$$

### Table 2

Now with a different table

$$\begin{array}{c|cccc} & Y =0& Y=1 \\ \hline X=0 & 0.3 & 0.3 \\ X = 1 & 0.1 & 0.3 \end{array}$$

The odds are the same

$$\frac{P(Y=1|X=0)}{P(Y=0|X=0)} = \frac{0.3}{0.3}= 1 \\ \frac{P(Y=1|X=1)}{P(Y=0|X=1)} = \frac{0.3}{0.1} = 3$$

The lift is changed

$$\frac{P(Y=1|X=1)}{P(Y=1)} = \frac{0.75}{0.4} = 1.875$$

• Hello! Thanks for your comment. Just a question, second line should be $P(Y|X=1) = logistic(a+b)$ or $P(Y|X) = logistic(a+bX)$, right? otherwise why/how did we get rid of the X variable? And as an extra note, just for clarity, logistic(a) is still the same as sigmoid(a) = $\frac{e^a}{1+e^a}$ Commented May 13 at 8:21
• @OscarFlores you can use $P(Y|X=x) = logistic(a+bx)$, and then fill in $x=1$. I have now removed the use of $X=1$ and $X=0$ and use $X$ and $\lnot X$ instead. Commented May 13 at 9:24
• How do we recover the $P(X)$ from the logistic regression? Usually we can do a statistical analysis on $a$ and $b$. So we can know the distribution of $a$, $b$, $P(Y|X)$, but do we assume $P(X)$ is constant in order to be able to estimate $P(Y)$? Commented May 15 at 7:51
• @OscarFlores logistic regression only computes $a$ and $b$. Logistics regression expresses the odds for $Y$ and $\lnot Y$ conditional on $X$. How often you have $X$ or $\lnot X$ is not important. The value of $P(X)$ is something that you can compute seperately. For different values of $P(X)$ and lift, you can have the same logistic regression coefficients $a$ and $b$. Commented May 15 at 7:54
• Ok ok... I got it. Thanks. Just one last comment, I think line 3 and 4 have a small typo, $P(X)$ and $P(¬X)$ are shifted, like, it should be $\frac{1}{P(X)+\frac{logistic(a)}{logistic(a+b)} P(¬X)}$ Commented May 15 at 10:03