I have datas of random events with gps and time variables. I would like to compute probabilities of an event happening at a given time and location. In the training set, we have about 1500 possibles areas and about 20.000 possible times. This gives 30.000.000 lines in my dataset and there is about 1000 events to learn from so there is a huge imbalance between 0's (absence of event) and 1's (event) to learn how to predict the 1's.

I have a few questions : - Is there a more clever way to work with this data than adding 30.000.000 lines with almost no information (but the 1's) ? - How would you modelize this problem ?

  • 1
    $\begingroup$ Choose a location (area) to work with where you have good data quality across time. Then sample records across your time dimension and build your model using it. $\endgroup$ Commented May 10, 2016 at 13:59
  • 1
    $\begingroup$ What exactly do you mean by predict the 1's? It may help if you could provide a few sample rows from your dataset and an example of what a prediction might look like, to give a better idea of what you are trying to do. You may want to look into multivariate kernel density estimation. $\endgroup$ Commented May 10, 2016 at 14:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.