1
$\begingroup$

I'm fitting a GLMM (mixed logistic regression) in R. I have five covariates. For model selection, I'm using glmmLasso() (in R) to determine which of the five covariates and their interactions should be put in the model.

Should outlier detection be done before or after model selection? What techniques would you recommend for outlier detection?

Improve this question
$\endgroup$
7
  • $\begingroup$ If this is logistic regression, there presumably are no outliers in your (0,1) response variable. Are there any technical issues in reliably measuring your covariates? What do you mean by outliers? And with only 5 covariates, why do you need to select a subset rather than incorporate all of them into your model? It also would help to know how many cases you have and in particular how many are in the least-frequent outcome class? $\endgroup$
    – EdM
    Commented Apr 30, 2016 at 21:08
  • $\begingroup$ I thought including all interactions would mean there are actually 15 covariates...so that model selection is needed...but maybe not? $\endgroup$
    – 193381
    Commented Apr 30, 2016 at 21:13
  • $\begingroup$ If you have on the order of 15 x15 = 225 cases in the least frequent outcome class, then you can reasonably try to use all variables and two-way interactions without too much danger of overfitting (15 of the least-frequent outcome cases per predictor is the rule of thumb). With fewer cases you could use ridge regression instead of LASSO as a penalization method that avoids overfitting yet keeps all predictors. Variables selected by LASSO can differ substantially among samples from the same population. Please say more about why you think you have outliers and the nature of the outliers. $\endgroup$
    – EdM
    Commented May 1, 2016 at 1:20
  • $\begingroup$ Oh, shoot, would I be overfitting if I were to use glmmLasso() to select variables? I saw that they used it for 3 variables in their paper: epub.ub.uni-muenchen.de/12268/1/TR108.pdf $\endgroup$
    – 193381
    Commented May 1, 2016 at 7:14
  • $\begingroup$ LASSO is a penalized method that keeps model coefficients lower in magnitude than they would be in a standard regression, to avoid overfitting. Ridge regression uses penalization of the squares of coefficients rather than their magnitudes. Both methods thus help avoid overfitting even when you start with more variables than are safe for standard regression. Choose the penalty in either method by cross-validation. Do not, however, select variables by LASSO and then put them back into a standard linear regression; you then lose the advantages of penalization. $\endgroup$
    – EdM
    Commented May 1, 2016 at 9:15

1 Answer 1

Reset to default
1
$\begingroup$

With logistic regression if your responses are 0/1 (rather than grouped and presented as fractions) what constitutes an outlier? Getting a 1 when the probability of 0 is high?

Responding more generally --

Should outlier detection be done before or after model selection?

You can't separate the two:

  • outliers are only outliers with respect to some model. An outlier for model A may not be remotely unusual for model B

  • model selection is impacted by points that would be outliers for some of the possible models

In a Bayesian context you could do both simultaneously, for example via MCMC. In a frequentist setting that's usually hard, but some kind of iterative process might work.

What techniques would you recommend for outlier detection?

I think that's much too general a question, but in any case my first thought would be to avoid detection (unless it's the outliers that are of central interest) and instead focus more on whether you can do something that's not especially sensitive to outliers. (Of the top of my head I doubt I can say anything sensible in that regard for GLMM)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Great to know that you don't have to always do outlier detection! I actually used cook's distances for mixed model and found no influential clusters. So I don't think further outlier detection is necessary. $\endgroup$
    – 193381
    Commented May 1, 2016 at 7:15
  • $\begingroup$ While you don't necessarily have to detect outliers, that doesn't mean you should replace it with nothing, of course. $\endgroup$
    – Glen_b
    Commented May 1, 2016 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.