Mixed-effects models with customer data: how do choices affect the model? Suppose I had a large sample of customer data from which I want to predict total amount of sales over a time period with predictor variables indicating:
-which sales channel did customers come from (e.g. internet, phone, store), which has a big impact on the variable to predict and the categories are not going to change.
-demographic data (e.g. age, gender, address) including zip code.
-detailed information for each zip code (e.g. average income, proportion of people with university degrees, etc.).
And I wanted to use them to both predict and make inferences on the total amount purchased over a period of time (not necessarily having the same model for both).
I’d like to ask what would be effects or the advantages/disadvantages of these approaches:
-Using a nested structure of state/city/zip code vs. appending the aggregated information from the zip code to each record (i.e. replacing zip code by columns indicating average income and such in that specific zip code).
-Treating the sales channel as a random effect vs. a fixed effect interacting with others, in terms of predictive accuracy.
Finally, how would you decide on classifying the sales channel as either a fixed or random effect?
 A: Sales channel should be fixed because:
a) the list is exhaustive. The sales channels in your sample cover all possible sales channels. 
b) You are not trying to infer to other, as yet unseen, sales channels.
c) The number of sales channels is small. It's hard to estimate the variance of a fixed effect without a reasonable number of levels - say 10 or more, and I prefer to see more.
The mixed effect model would assume that the sales channel effect is normally distributed -- in other words, it sort of randomly adjusts the outcome, but in symmetric and controlled ways. Suppose that phone purchases are WAY lower than purchases made in stores and over the Internet because people can't browse on the phone and make impulse purchases. A mixed model would tend to adjust the impact of phone sales upwards to make them more typical of other channels. Whereas with fixed effects, the outlier status of phone sales would be highlighted.
Anyway, I would certainly use fixed effects in your situation if you only have the 3 sales channels that you mention.
The thing about predicting mixed effects models is that they predict the mean of future observations on the basis of the fixed effects, but adjust the size of the confidence intervals to take into account the additional variability caused by the random effects. You can estimate the value of the random effects for the data you have ... so you could estimate the purchasing effect of zip code 12345, say, if it's in your sample. The effect would be shrunk towards zero, in comparison to the fixed effect you would have from treating zip code as a fixed effect. But if you wanted to predict sales from an unseen zip code, the predicted mean would depend on the fixed effects alone. However, if you included average income, and knew the average income of some unseen zip neighbourhood, you could use that information to refine your prediction.
So good idea to include average income.
