# Predictive Modeling - Should we care about mixed modeling?

For predictive modeling, do we need to concern ourselves with statistical concepts such as random effects and non independence of observations (repeated measures)? For example....

I have data from 5 direct mail campaigns (occurred over the course of a year) with various attributes and a flag for purchase. Ideally, I would use all this data combined to build a model for purchase given customer attributes at the time of the campaign. The reason is that the event of purchase is rare and I would like to use as much information as possible. There is a chance that a given customer could be on anywhere from 1 to 5 of the campaigns - meaning there is not independence between the records.

Does this matter when using:

1) A machine learning approach (e.g. tree, MLP, SVM)

2) A statistical approach (logistic regression)?

**ADD:**


My thought about predictive modeling is if the model works, use it. So that I have never really considered the importance of assumptions. Thinking about the case I describe above got me wondering.

Take machine learning algorithms such as a MLP and SVM. These are used successfully to model a binary event such as my example above but also time series data that are clearly correlated. However, many use loss functions that are likelihoods and derived assuming the errors are iid. For example, gradient boosted trees in R gbm uses deviance loss functions that are derived from the binomial (Page 10).

• It will matter for statistical approaches that assume independence between the records, because you're then dealing with repeated measures. – Michelle Feb 7 '12 at 22:24
• Seems to me one of the main differences between machine learning focused on prediction and statistics focused on inference is exactly what you say, B_Miner. Machine learning is more concerned with what works while traditional statistics pays special attention to assumptions. In both cases you need to be aware of assumptions/properties of your approaches then make an informed decision whether they matter or not. You might be fooling yourself in predictive modeling about whether your model works if you don't understand the assumptions/properties of the approach. – Anne Z. Feb 8 '12 at 12:44
• @AnneZ.If you follow the recommended validation approach of training-, test- and validationset (all samples big enough) in predictive modelling and you find something that works, does one has still to bother if the underlying assumptions are met ? I certainly do not recommend mindless application of ML, I was just wondering ... – steffen Feb 10 '12 at 10:02
• In this context, the paper "Statistical Modeling: The Two Cultures" might be interesting, discussed in the third crossvalidated journal club – steffen Feb 10 '12 at 10:05