For predictive modeling, do we need to concern ourselves with statistical concepts such as random effects and non independence of observations (repeated measures)? For example....
I have data from 5 direct mail campaigns (occurred over the course of a year) with various attributes and a flag for purchase. Ideally, I would use all this data combined to build a model for purchase given customer attributes at the time of the campaign. The reason is that the event of purchase is rare and I would like to use as much information as possible. There is a chance that a given customer could be on anywhere from 1 to 5 of the campaigns - meaning there is not independence between the records.
Does this matter when using:
1) A machine learning approach (e.g. tree, MLP, SVM)
2) A statistical approach (logistic regression)?
My thought about predictive modeling is if the model works, use it. So that I have never really considered the importance of assumptions. Thinking about the case I describe above got me wondering.
Take machine learning algorithms such as a
MLP and SVM. These are used successfully to model a binary event such as my example above but also time series data that are clearly correlated. However, many use loss functions that are likelihoods and derived assuming the errors are iid. For example, gradient boosted trees in R
gbm uses deviance loss functions that are derived from the binomial (Page 10).