Cross-Validation driven outlier removal Can I turn the outlier removal problem into a model-selection problem and use cross-validation to solve it?
Take a situation similar to this question: you have a mixed model you want to fit against data. However the data-set contains a number of outliers (due to poor recording, poor explanation of the experiment to the subject and so on).
Unfortunately there are many methods to remove outliers in the literature; a priori all equally arbitrary.
What I would like to do is to use cross-validation to help me select the right outlier removal protocol. 
To simplify things, imagine that we decided to remove all independent variables who are $z$ standard deviations away from their mean and we have to pick $z$ between $\{2,2.5,3\}$ (this is purely as an example).
Is it possible for me to run outlier removal + regression fit on testing data and use the mean square prediction error on (unfiltered) training folds to choose $z$? 
Moreover, since there is no outlier removal on the testing data, should I use a different prediction error form than sum of squares?
 A: One approach is Frank Harrell's in his book Regression Modeling Strategies (2nd edition, 2015) where he discusses cross-validation and outlier detection (p.25 of 1st ed.) in the context of nonparametric, loess smoothers:

After making an initial estimate of the trend line, loess can
  look for outliers off this trend.It can then delete or downweight those
  apparent outliers to obtain a more robust estimate.

Harrell's treatment continues with discussion of other approaches such as Friedman's "super smoother," Hastie and Tibshirani's GAMs model and other local regression methods.
Choice of MSE vs MAD is pretty much a function of the analyst's preferences and training.
A: My approach to a problem like this would be to express my mixed-effects model in the form of a hierarchical Bayesian model [1], and then exploit the flexibility of Bayesian models to introduce a latent 'outlier' variable that could be used to discount (or eliminate) the contribution of 'outlier' data to the model likelihood.
It appears from your OP that you have some substantive prior knowledge of what causes outliers in your data. Such knowledge could be expressed objectively in a Bayesian model of the kind I'm hinting at, thereby dealing nicely with the 'cherry picking' problem Greg Snow described in his answer to the post you linked above.
Also, if any of your priors include knowledge of directionality, then you might have a problem of censoring. But taking advantage of the residual information in a tiny handful of 'outliers' (you haven't indicated how common your outliers seem to be) would probably be more trouble than it was worth. 
[1]: Gelman, Andrew, and Jennifer Hill. Data Analysis Using Regression and Multilevel/hierarchical Models. Analytical Methods for Social Research. Cambridge ; New York: Cambridge University Press, 2007.
