Let's say I have a dataset about passages of cars on a road. The dataset contains information about time, driver, car, weather, and most importantly whether the car was involved in an accident.

Of course the number of accidents is very small compared to the total number of cars passed on the road (let's say around 1/10,000). The dataset contains around 10,000,000 passages, with roughly 1000 accidents.

I would like to create a model to predict the probability of a car being involved in an accident given information about time, weather, driver etc.

A possibility I thought about would be to calculate and use as features the ratio of accidents for each population (e.g. 0.000009 for male young drivers driving with good weather in the morning), but given the relatively small number of accidents I think this approach would be unreliable.

Another possibility could be to use the "raw data" as features and to set the probability of accidents to train the model to 1 when accidents occurred and to 0 when they didn't, but I am not sure I could actually interpret the output of the regressor as the probability of the event happening.

Would the latter be a good approach, or are there any other possible ones I don't know about?

  • 2
    $\begingroup$ Look into logistic regression. $\endgroup$ – kjetil b halvorsen Apr 21 '17 at 22:48
  • 2
    $\begingroup$ Look into poisson regression. (Nicely done question.) $\endgroup$ – rolando2 Apr 22 '17 at 12:28
  • $\begingroup$ @rolando2 isn't poisson regression used for count data (in contrast to binary data, for which logistic regression is used)? If not, could you please point to some resource about using poisson regression to estimate the probability of binary events? Thank you! $\endgroup$ – MDG Apr 22 '17 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.