I'm setting up a logistic regression that models a probability $\mathbb{P}_t$ of certain events. The probability is changing over time and I want to add proportions of past observations that estimate this probability. Suppose I have a proportion $q = \frac{\text{#positives}}{\text{#total}}=\frac{b}{c}$ that estimates $\mathbb{P}_t$ if times are relatively close. How do I add it as a feature in a logistic regression model?

My guess

Because the logistic regression model is

$$ \log \frac{p}{1-p} = x \cdot \beta,$$

1) my intuition says add $\log q$ and $\log(1-q)$. Then our logistic regression model would clearly encompass the baseline model of $p=q$. In the case where the coefficient of $\log q$ is $+1$ and the coefficient of $\log(1-q)$ is $-1$ and other coefficients in $\beta$ are $0$.

2) Another option, remembering that $q=\frac{b}{c}$, is to express

\begin{align} \log{q}&= \log{b}-\log{c}\\ \log{1-q}&=\log \frac{c-b}{c}=\log{(c-b)} - \log{c}. \end{align}

Here I would be tempted to add $\log{(b + 1)}, \log{(c + 1)}, \log{(c - b + 1)}.$ as features.

  • 2
    $\begingroup$ One thing you may want to do is "stabilize" the proportion to handle the log transform, something like $\frac{b+a}{c+2a}$. You may also want to add the logit itself as the predictor $\log\left(\frac{b+a}{c-b+a}\right)$. This is simpler than adding 3 predictors. $\endgroup$ – probabilityislogic Aug 2 '13 at 21:58

It sounds like you're trying to estimate a Markov transition matrix for forecasting binomial event data, such as enrolling in a certain medical insurance option or presence of open sores in patients with positive diagnoses of herpes.

Doing this, you definitely want to apply some sound theory. For instance, are you using the cumulative proportion of past observations? Then caveat emptor, your logistic regression model will place equal weight on observations at time 2 (for which only 1 prior observation is used to predict "prior prevalence") versus observations drawn out at time 10 from which you have a much more stable weight. You might consider weighted logistic regression, hierarchical modeling, or random forests (for estimating decision trees).

If you were to incorporate prior prevalence as an additive effect in modeling the log-odds of a positive outcome, then the choice of $\log(p)$ and $\log(1-p)$ seems rather unintuitive and restrictive to me. In my mind's eye, adjusting for the prior odds alone seems sufficient, since any increase or decrease in prior outcome odds are most likely to be either positively or negatively related to the odds in the observed outcome. Alternately, if the linear effect is not sufficient, I would choose a more robust polynomial based interpolating spline approach to handle heterogeneity in the relationship between prior prevalence and positive outcomes.

  • $\begingroup$ I am not modelling stateful data, rather interaction (events/moments) data. The choice of $\log(q)$ and $\log(1-q)$ just enables the past log odds to be taken into account directly. It is the same logistic transformation applied to past interactions to predict future interactions. What is the alternative in formulae? $\endgroup$ – Peteris Aug 1 '13 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.