I'm doing a classification task (just like ad ctr prediction), the data is generated day by day, for example, data from
06-15 is collected, and I split the whole data into 2 parts, the
06-12 as training set, and the
06-15 part as test set.
Over the training set, I build a
Logistic Regression model, and I also do cross-validation on this training set when I build the model, the cv score is good, then I re-train the model over the whole training set.
Now I do prediction over the test set using the model I built above, however the result is very bad.
What would be the problem here in my case? I'm sure that there is no strong temporal dependency between the test set and the training set. How should I debug this?