Low accuracy in out of time validation I prepared a model which had very good accuracy (80.5%) on my out of sample data. However when I ran that model on a population which is some 6mths old the accuracy went down to abysmal 33%. I am talking here about percentage detection of event (say defaulters). So right now my model is only detecting 33 out of 100 defaulters in the out of time dataset. 
Please suggest what can be the possible reasons behind this. How do I go about improving this? It'd be tough to defend this accuracy before the client. God forbid, if required to defend this accuracy before the client, how could this be justified?
 A: I'm assuming that you expect that the other data set should have similar characteristics to your original data set. I only consider myself a beginner in this area, but it sounds likely that you "over-fitted" your model to the sample data. This means fitting to random noise in the data, as though it is a real effect that would be observed on a new data set. A likely cause is having too many parameters relative to the sample size.
You haven't provided much detail, but possible solutions may be reducing the number of parameters in your model, and/or the use of a shrinkage method (which I know very little about). The bootstrap may be useful to validate your model (validate in the rms package in R).
These things are covered in the book "Regression Modeling Strategies" by Frank Harrell.
A: The most obvious culprit for your problem is probably spurious relationship. You identified relationships which deemed significant for certain period of time, but they are not significant for all periods of time. Lucas critique might apply too. 
When dealing with models of economic activity it is always prudent to define the time boundaries of the model (i.e. for which time period the model is applicable) or include time in the model. It seems you are building your model on snapshots at specific time-periods, try using panel-data framework, this will help you to see which covariates remain significant for all time periods.
A: Could you provide information about the nature of data (cross-section, time-series, panel...)? In any case, it seems to me that one pssible problem is that there is a time trend that you are not take into acount. Another possibility is just that the data is not stationary, i.e, th past does not resemble the future at all and as time goes by your accuracy will decrease.
Finally, you have to think in relative terms. With no model, it seems that you'd predict only 3.79% of defaulters. Assuming you predict on a random bases, your accuracy would be very low (probaly near zero). On the other hand, with your model you'd predict with accuracy of 30%, which is much better. For instance, with 1000 people, without model you would say I expect only 30 of them to default, but they are equally likely to default. With your model, you will say: person 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30 will default (and only one third of them will actually default). This is much better!
However, take a look not only at the true-positive rates, but also on false-positive reates of your model and false-negatives. A ROC curve may be a good way to look at this and calibate your model better to increase whatever the client wants (true-positive rates, true-negatives).
