Model instability in data mining. When it is big enough to discredit a model and how to measure it? Let's say I have two models. One has cumulative lift on test data 4.322578, second 2.84488. The only advantage of the second over the first consists in the quality of having the cumulative lift curve nearly identical for both train and validate sets. 
For the first model the curve of cumulative lift looks as you can see below.

Sorry for non-english elements on the plot, but that's the only SAS Enterprise Miner version I have access to.
Mean standard error for the model from the plot - the first one - for train, validate and test sets are: 0.038389, 0.04999 and 0.055314.
For the second one: 0.070393, 0.071889 and 0.07727.
First model seems to be obviously better in terms of accuracy. But at the same time it is less stable, more sensitive to data changes. 
My question is whether those stability issues are here big enough to discredit that model in favor of the second one? How to measure instability and at which point it renders a model unusable?
If it is important, the models in question are neural networks.
 A: Maybe a bit late to the party, but you can directly measure how stable the model or its predictions are with respect to slight changes in the training data. 
Resampling validation looks at a (large) number of surrogate models that differ only in that a (small) fraction of the training cases is exchanged. Thus, if you look at the prediction this set of models gives for the same test case, you can measure the (in)stability of the predictions. 
Here's our paper which explains the idea in more detail:
Beleites, C. & Salzer, R.: Assessing and improving the stability of chemometric models in small sample size situations, Anal Bioanal Chem, 390, 1261-1271 (2008). DOI: 10.1007/s00216-007-1818-6
As to when does instability become a problem I don't see how this can be answered without the context of your application. 
A: If the model is build for prediction, then the choice of an optimal model should be based on the test data. In your model, I don't quite understand the difference between "validation" and "test" data. Is validation refering to cross-validation?
It appears that the first model provides a better prediction over the second. The difference between training and testing is expected -- the test error will almost always be larger. Beyond that, it's difficult to conclude that the model is 'unstable' based on the magnitude of the change.
