Predicting Negative Binomial response when some observations contain no success

Question

Problem description:

I am attempting to model the number of attempts required in order to make event $Z$ happen. Each day we make $k_i$ attempts with setting $X_i$ (a vector) in an effort to make $Z$ happen for object $i$, for $i$ from 1 to $m$. In other words, we are making multiple attempts with one setting (these settings affect the probability each attempt causes $Z$ to happen), then multiple attempts with another setting, et cetera. However, once we start making attempts with a new setting the previous set of attempts have no effect, because now we've given up on making $Z$ happen (or we've made it happen already) for object $i$ and are attempting to make $Z$ happen for object $j$.

Context:

The attempts have a cost and the events have a payoff. We are wanting to predict the number of attempts needed so that if a large number of attempts is predicted to be needed we can know not to make any attempts, because it won't be worth it.

Issue:

My first thought was to just model this with Negative Binomial regression, and the target "count" would be number of attempts required. The problem is that most of the time, event $Z$ doesn't actually happen, so the count for that observation is missing (but known to be greater than $k_i$).

Answer 1

Let $f(y)$ be pdf and F(y) be CDF of negative binomial distribution. Let $n$ be # of Z happened, and $m$ be the # of Z not happened.

Then log-likelihood can be write as: $$\mathrm{LogL} =\sum_{i=1}^n \log f(y_i) +\sum_{j=1}^m\log (1-F(y_j))$$ where $y_i$ means at $y_i$-th trail, Z happened, $y_j$ means Z did not happen till $y_i$-th trail and no more trail.

Answer 2

You should check out zero inflated models. They theorize that the excess zeroes come from a separate population and try to fit a likelihood to that and then a separate likelihood for the attempts.[Here's a handy page on the zero inflated negative binomial] you could fit a model where your conditions are predictors and the model should be able to tell you their effect on successes as well as their effect on getting zero. (If I've understood you correctly)