1
$\begingroup$

I would like some advice on modeling rare events that arises from a continuous variable crossing a threshold.

I have a data set consisting of a continuous response $Z_t$ and continuous regressors $A_t, B_t, \dots$ and would like to model when $Z_t$ crosses some threshold $k$ (i.e., when $Z_t > k$). A logistic regression works reasonably well, but there are a few problems: the events are rare (I have about 1200 observations with less than a dozen events) and I would like this to back test well (this means estimating the model on observations where $t_1<t<t_2$ and then sliding $t_1,t_2$ forward in time). Given the rarity of events, during back testing there will be many $t_1,t_2$ with few, if any, events. Furthermore, given that there is a continuous variable, $Z_t$, underlying the events, it seems that focusing on just the events results in some information being discarded (such as a "near miss", where $Z_t$ is only slightly less than $k$).

A natural approach would to directly model just $Z_t$ using a linear model. This does not work very well, probably because away from $k$, $Z_t$ behaves in a way that is not well modeled by a linear model. So, loosely speaking, it seems that what I want is a model that is similar to a linear model but is "tuned" to pay attention to $Z_t\approx k$.

For example, here is one concrete approach: convert $Z_t$ into a value between 0 and 1 (via a logit function, say; map $Z_t=k$ to $0.5$) and then use a fractional logistic regression.

What are some modeling strategies for a situation such as this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.