# Coefficient / model averaging to control for exogenous circumstances in prediction

I'm interested in exploring statistical models (or modifications thereof) designed to handle a specific type of problem. Due to my ignorance of statistical terminology, I can only describe this type of problem by (contrived) examples:

Suppose we're interested in estimating the likelihood that a given cell phone customer will drop his/her service (churn) over the course of the next month (October). We are given cell phone user data from the last three months (July, August, September).

In estimating churn probability for October, we'll want to make use of a model that emphasizes inputs that are consistently predictive of churn each month, suppressing the importance of those whose coefficients vary heavily on a month-by-month basis. For example, "market" may be consistently predictive of churn, but an externality specific to one of the training datasets (e.g., marketing of new phone by a competitor in a specific market in August) may affect a coefficient in a way that won't necessarily apply to October. The market coefficient should be corrected/compressed/averaged when predicting October's churn likelihood.

Here's a less contrived example. In estimating the likelihood that a voter will vote in the 2012 election, we may use voter-level data from the 2000, 2004, and 2008 elections to train a logistic regression on turnout. If "party" is an input, 2000 and 2004 coefficients may differ significantly from 2008 party coefficient due to exogenous circumstances (a unique political environment). In estimating 2012 vote likelihood for a given voter, we'll want to trim/compress/average the party coefficient to "smooth out" election-year-specific differences that can't be easily quantified.

Can anyone point me in the right direction? Any guidance on how to better ask this question? What terminology should I be using? Thanks in advance for your help.

• One of the best ways to improve a forecast is to average different forecasts (from different models, or from different experts or both...). You may want to look around on forecastingprinciples.com or look through past issues of the International Journal of Forecasting. – Stephan Kolassa Oct 21 '10 at 13:02

I am not sure you need any special tricks as long as the relevant factors are captured by the model. To keep things simple I will discuss the issue in the context of linear regression. The same intuition carries over to the time series setting.

Suppose, that you want to predict the monthly sales of cell phones for brand X and to illustrate the ideas suppose that you use linear regression to do so. The issue you have is that there are some factors that impact sales every month (e.g., weather of the month to take a silly example) and some factors that are idiosyncratic to a month (e.g., launch of a new cell phone). You wish to account for both factors consistently.

Denote the factors that impact sales every month by $C_m$ (where 'C' stands for common and 'm' for month) and idiosyncratic factors by $I_m$ (where I stands for idiosyncratic and m for month). Then, your model would be:

$S_m = \beta_c C_M + \beta_i I_m + \epsilon$

In the months where there is no idiosyncratic factors $I_m$ would be 0 and hence the impact on sales would be captured by $\beta_c$ and in the months when you do have idiosyncratic factors the impact on sales would be $\beta_c + \beta_i$ with $\beta_i$ presumably being negative if the idiosyncratic factor dampens sales for brand X. Thus, the coefficients 'automatically' adjust themselves provided all the necessary factors are present in the model.

Thus, it seems to me you do not have to worry about the issue you raise if you do account for all factors whether they are idiosyncratic to a time period or common across time periods.

Edit in response to comment

Your example of concert and increased cell phone usage by young consumers is an example of an interaction effect. To put it in different words your example is saying that the effect of the concert on cell phone use is higher for younger consumers rather than older consumers. Basically, your model is saying the following:

Minutes = beta1 age + beta2 * concert + beta3 * age * concert + error

Thus, the beta3 parameter will capture the differential impact how younger/older consumers respond to the concert.

• Thanks for the answer! However, I'm interested in modeling individuals within groups where group-level effects are difficult to quantify or anomalous and can't easily be included in future predictions. To alter your example a little, suppose we're predicting minute usage for individual customers over the next month using the last three month's of data. Suppose that while customer age is a significant predictor of usage, a local concert last month made young users unusually likely to make phone calls. The 'age' coefficient may be unnaturally large and skew predictions. Does this make sense? – danpelota Oct 18 '10 at 13:20
• @danpelota That is an example of 'interactions'. I will edit the answer to clarify. – user28 Oct 18 '10 at 14:16
• Thanks Srikant. Since the group-level effects to which I'm referring often unique, complex, and erratic, I'm not sure I'll be able to quantify them with interaction terms. However, your answer has led me to approach my problem from a different angle and refine my question. I'll restate in more formal terms once I've more clearly defined the problem. – danpelota Oct 20 '10 at 14:30
• I really think you should use a multilevel model. A varying intercept and varying slope by month is exactly what you want. Take a look at Gelman and Hill's book Analyzing Regresssion with multilevel models or look here at stats exchange about multilevel logistic. It's fairly easy to run a simple multilevel model in R with package lme4. See this thread for instance: stats.stackexchange.com/questions/9429/… – Manoel Galdino Aug 8 '11 at 14:19