3
$\begingroup$

I have noticed that there is an interesting connection between two (apparently different) measures. I am under a market basket analysis framework (aka frequent itemset mining, both are common names) , where all the variables are hot-encoded. So our predicted variable Y and our predictor variable X are both dummy variables which represents the existance or non-existance of a product in a transaction.

The first measure 'lift' comes from apriori algorithm.

it is defined as $$ \frac{confidence(A \rightarrow B)}{support(B)} = \frac{P(B|A)}{P(B)}$$ which is basically $$ \frac{P(A \cap B)}{P(A)P(B)}$$

The second measure comes from a univariate logistic regression.

$$logit(Y) = a + bX + e_i$$

My doubt is in the logistic regression part. a is the intercept and b represents the difference in log odds.

Both measures are symmetric, when regressing Y on X or X on Y (or when using apriori) I still get the same lift and b. But they have different scales. How is this related? Is there a form I can translate from b values in a logistic regression to a lift equivalent?

Any hints or bibliography where I can find more information is highly appreciated.

Edit:

lift and logistic_coefficients

Here I attach a small sample. These represent some rules extracted by the apriori algorithm. Column Antecedent represents the product A, Column Consequent represents product B. Lift and logistic_coefficients are our variables of interest in this case, and I am trying to figure out how are they related and if we can map from one to the other one.

(In case someone is interested, here's the link for other results)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The lift is a combination of the difference in the odds (related to the coefficient $b$), and the probability for the cases $x=0$ and $x=1$.

We can write $$P(Y) = logistic(a) P(\lnot X) + logistic(a+b) P(X)$$ and $$P(Y|X) = logistic(a+b) $$ making

$$\begin{array}{} \frac{P(Y|X)}{P(Y)} & =& \frac{1}{P(X)+\frac{logistic(a)}{logistic(a+b)} P(\lnot X)} \\ &=& \frac{1}{1+\frac{e^{-a}(e^{-b}-1)}{e^{-a}+1}P(\lnot X)} \end{array}$$

An example below gives the same logistic regression but different lift.

Table 1

$$\begin{array}{c|cccc} & Y =0& Y=1 \\ \hline X=0 & 0.1 & 0.1 \\ X = 1 & 0.2 & 0.6 \end{array}$$

The odds are

$$\frac{P(Y=1|X=0)}{P(Y=0|X=0)} = \frac{0.1}{0.1}= 1 \\ \frac{P(Y=1|X=1)}{P(Y=0|X=1)} = \frac{0.6}{0.2} = 3 $$

The lift is

$$\frac{P(Y=1|X=1)}{P(Y=1)} = \frac{0.75}{0.7} \approx 1.071$$

Table 2

Now with a different table

$$\begin{array}{c|cccc} & Y =0& Y=1 \\ \hline X=0 & 0.3 & 0.3 \\ X = 1 & 0.1 & 0.3 \end{array}$$

The odds are the same

$$\frac{P(Y=1|X=0)}{P(Y=0|X=0)} = \frac{0.3}{0.3}= 1 \\ \frac{P(Y=1|X=1)}{P(Y=0|X=1)} = \frac{0.3}{0.1} = 3 $$

The lift is changed

$$\frac{P(Y=1|X=1)}{P(Y=1)} = \frac{0.75}{0.4} = 1.875$$

Improve this answer
$\endgroup$
6
  • $\begingroup$ Hello! Thanks for your comment. Just a question, second line should be $P(Y|X=1) = logistic(a+b)$ or $P(Y|X) = logistic(a+bX)$, right? otherwise why/how did we get rid of the X variable? And as an extra note, just for clarity, logistic(a) is still the same as sigmoid(a) = $\frac{e^a}{1+e^a}$ $\endgroup$
    – Oscar Flores
    Commented May 13 at 8:21
  • 1
    $\begingroup$ @OscarFlores you can use $P(Y|X=x) = logistic(a+bx)$, and then fill in $x=1$. I have now removed the use of $X=1$ and $X=0$ and use $X$ and $\lnot X$ instead. $\endgroup$
    – Sextus Empiricus
    Commented May 13 at 9:24
  • $\begingroup$ How do we recover the $P(X)$ from the logistic regression? Usually we can do a statistical analysis on $a$ and $b$. So we can know the distribution of $a$, $b$, $P(Y|X)$, but do we assume $P(X)$ is constant in order to be able to estimate $P(Y)$? $\endgroup$
    – Oscar Flores
    Commented May 15 at 7:51
  • 1
    $\begingroup$ @OscarFlores logistic regression only computes $a$ and $b$. Logistics regression expresses the odds for $Y$ and $\lnot Y$ conditional on $X$. How often you have $X$ or $\lnot X$ is not important. The value of $P(X)$ is something that you can compute seperately. For different values of $P(X)$ and lift, you can have the same logistic regression coefficients $a$ and $b$. $\endgroup$
    – Sextus Empiricus
    Commented May 15 at 7:54
  • $\begingroup$ Ok ok... I got it. Thanks. Just one last comment, I think line 3 and 4 have a small typo, $P(X)$ and $P(¬X)$ are shifted, like, it should be $\frac{1}{P(X)+\frac{logistic(a)}{logistic(a+b)} P(¬X)}$ $\endgroup$
    – Oscar Flores
    Commented May 15 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.