I am trying to predict a binary response based on multiple binary independent variables, some categorical data, and some continuous data. My data set looks similar to:

Y B1 B2 B3 B4 B5 B6 Ca1 Ca2 Co1 Co2
1  0  1 0  1  0   0  A   A  1.5 1.7
0  1  1 0  0  0   0  B   C  2.3 1.1

Unfortunately, my n is only 119. Of these, y=1 in only 4 observations. I am assuming that I am therefore underpowered to detect much of anything significant.

My end goal is to try to build a model whereby I can assign a weight to each variable to create a cutoff point where an 80% probability of y=1 exists. So for example, B1 is assigned 5 points, B2 is assigned 1 point, and so on. Then when added up, if points > 15, then probablity > 80% that y = 1. This is not dissimilar to the Canadian Head CT rule.

What is the appropriate statistical methodology to undertake this kind of endeavor?


Would the methodology described in this paper describe what you are interested in, Dylan?

If yes, the idea is that you have to build a binary logistic regression model of the form:

log(odds that y = 1) = beta0 + beta1*B1 + beta2*B2 + ... + beta10*Co2

The weights you are interested in can be estimated from the data and are the estimated values of the beta coefficients in the above equation:

b0, b1, b2, ..., b10. 

The article mentions that these estimated values could be rounded for convenience so they are easier to use by physicians, though that rounding should be done with care and used cautiously. The better thing to do is to use the actual estimated values as input for a web-based app, say. This app would estimate the probability p that y = 1 given the values of the predictors using an equation of the form:

p = exp(lp)/(1 + exp(lp)) 

where lp (i.e., linear predictor) is equal to:

lp = b0 + b1*B1 + b2*B2 + ... + b10*Co2.

The challenge in your case is that y = 1 is a rare event so you have to fit the binary logistic regression model in a way that takes this into account. There are multiple modalities of accomplishing this. See, for example, here.

In practice, once you are able to estimate the model, if you select a patient at random from your target population and know that patient's values for B1, B2, ..., Co2, then you can plug them into the above two equations to estimate p (expressed as a proportion). If p > 0.8 (or 80%, an a priori cut-off value), you can classify that patient as having y = 1; otherwise, you can classify that patient has having y = 0. The target population is the population of patients represented by the ones included in your model.

The article I linked to discusses the need to assess the performance of your model and also to perform internal validation of the model using perhaps bootstrapping. Of course, you can use model selection to decide what predictor variables should be included in the final model if not all predictor variables are clinically important and/or the rare event situation may hamper your efforts to fit a full model.

I hope I have provided enough food for thought here for you to determine what your next step should be. By the way, I am a Manchester United fan myself and am looking forward to watching the West Ham game tomorrow. ⚽️

  • 1
    $\begingroup$ This seems to be exactly what I want to do! The challenge for me now (as a physician with no particular statistical training) will be to slowly work through this process with my raw data. Thank you for providing me with such great sources. It hurts to see Liverpool and Man City in 1 and 2! $\endgroup$ Sep 29 '18 at 2:56
  • $\begingroup$ Mourinho out and then we can start thinking about a better placement in the Premier League. 😛 $\endgroup$ Sep 29 '18 at 3:09
  • $\begingroup$ @Isabella - Not sure any model would've predicted Man United's defeat 😜 $\endgroup$
    – n1k31t4
    Oct 2 '18 at 5:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.