1
$\begingroup$

I have a dataset with event data. It has a date of start and date of finish vareiable. I need to predict time remaining until an event finishes. The problem is that I can't use events in future time to predict events before that date. That is why I need to somehow split the data into training set with only events that finish before the events in the testing dataset start and still maintain a nice 75/25 or 80/20 split. Any ideas?

Thanks!

Some clarification of the problem:

Events are actually aggregated cases with one-hot encoded columns of different type of events as counts of occurance. The problem is to predict the actual time remaining until the end of an event. The reason for this particular split, is that you can't use events that happen in the future, to predict the remaining time of cases (and therefore events belonging to this case). If you do that, it is assumed that they are independent, but they are actually not.

$\endgroup$

2 Answers 2

0
$\begingroup$

As Henry pointed out, the real world is messy. You may find examples like Henry's and others that make your task not as easy as it sounds. Also, the business problem may not be optimal splitting with test only to the newest events. Perhaps in the financial world, you want to test your model against the latest data and recession data where the recession happened before your training data time period. Or maybe a critical date such as interest rates started increasing.

However, assuming everything is tidy, you can use a cumulative sum then split at a date where the sum is appropriate.

Here is some R code for the mechanics assuming tidy. This assumes events start and finish in the same month and can only split on a month.

finish_month <- c("Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
num_finish_in_month <- c(5, 10, 5, 10, 5, 10, 5, 10, 5, 10, 5, 10)
total_finish = sum(num_finish_in_month)
cum_sum <- cumsum(num_finish_in_month)
# split close to 80/20 without going over
cut_off <- max(which(cum_sum < (total_finish * 0.8)))
use_month <- finish_month[cut_off]
print(use_month)
#test it
num_finish_before <- sum(num_finish_in_month[1:cut_off])
num_finish_after <- sum(num_finish_in_month[(cut_off+1):length(finish_month)])
print(num_finish_before/total_finish)
print(num_finish_after/total_finish)
print(num_finish_before+num_finish_after)

[1] "Sep"
[1] 0.7222222
[1] 0.2777778
[1] 90
$\endgroup$
0
1
$\begingroup$

It depends on the data and your reason for saying data into training set are only those events that finish before the events in the testing dataset start

One approach might be to take the latest date which puts $20\%$ of events starting later and make those your test set. Put it to one side until the final test, in the usual way

You then have a training set which has a combination of (a) events which started and finished before the critical date and (b) events which started before the critical date but finished after it. You should if possible include the events in (b) in your training, treating them as right-censored at the critical date

This censored data could complicate your training, but failing to include it could bias your results by making it appear that events in the training set have shorter intervals on average than the underlying population distribution

$\endgroup$
2
  • $\begingroup$ Thanks so much for the response. For (b) you mean events which started before the critical date, but are not necessarily finished by then, I presume? Then yes, a solution would be even therefore removing the ones that have not finished at the critical date. I also added a bit of clarification in the original post. $\endgroup$
    – Emiliyan
    Feb 24, 2019 at 14:24
  • $\begingroup$ @Emiliyan - As I said, I think if you remove the cases which started before the critical date but ended after, then you may bias the results. You may also bias the test set results if you have already removed those cases which have not finished as of today's date $\endgroup$
    – Henry
    Feb 24, 2019 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.