4
$\begingroup$

I've been told not to trim the dependent variable in a regression, but I don't know why. It makes sense that I shouldn't select my sample based on the outcome, but what assumption does this violate? Is their a theoretical reason why I shouldn't do this? Thanks!

Update: By "trim" I mean discard outliers. On the right hand side it is common (at lest in financial economics) to discard or trim observations at 0.5% to 1% in either tail. I've been told that doing the same on the left hand side is taboo. But I'm not sure exactly why.

I don't have a specific problem in mind, I just realized that I don't know the real why, other than you shouldn't pick you sample based on the outcome.

$\endgroup$
  • $\begingroup$ Terminology sometimes varies; can you describe your data, your goals, & what it is exactly that you want to do? $\endgroup$ – gung - Reinstate Monica Apr 5 '13 at 15:13
  • 4
    $\begingroup$ Automatic trimming in either tail is a bad idea except where extensive experience with a specific data-generation process supports it--and even then it's a bad idea. But think about the regression situation: when there really is a linear relationship, the 1% are there in the tails because they are the responses at extreme (or almost extreme) values of the independent variables. That can be where you want to characterize the regression most accurately. Thus, trimming dooms you inevitably to using the middle values of your data to extrapolate into the extreme values: yet another bad idea. $\endgroup$ – whuber Apr 5 '13 at 15:35
  • 4
    $\begingroup$ If it's the standard of the industry, I worry about the industry. @whuber said it well. If you think you need to trim, think instead of finding the appropriate transformation of $Y$ or of using a more robust model such as the proportional odds model or quantile regression. There are many other robust regression models. $\endgroup$ – Frank Harrell Apr 5 '13 at 16:50
  • $\begingroup$ Right-hand side / left-hand side of what? $\endgroup$ – Scortchi - Reinstate Monica Apr 5 '13 at 20:30
  • 1
    $\begingroup$ thanks for the clarification about terminology @richardh - when I read your question first I assumed you meant for some reason it was common to discard large outliers (right hand side of the distribution) but not small ones (left hand side of the distribution). $\endgroup$ – Peter Ellis Apr 7 '13 at 5:42
10
$\begingroup$

There is a simulated data set called outliers in the TeachingDemos package for R. If you remove the "outliers" using a common rule of thumb, then relook at the data and remove the points that are now outliers and continue until you have no "outliers" you end up trowing away 75% of the data as "outliers". Are they really unusual if they are the majority of the data? The examples on the help page also show using this data for a regression model and how throwing away half of the data as outliers does not make much difference.

This is intended as an illustration against using automated rules for throwing away data.

Actually the discovery of penicillian was an outlier, consider what the world would be like if that data point had been discarded instead of investigated.

There are more acceptable routines such as M-estimation or other robust regression techniques that downweight unusual observations rather than throwing them out.

$\endgroup$
3
$\begingroup$

Rejecting to predict particular $X$s that do not fit your specification can be seen as part of the model:

$ Y = \begin{cases} f (X) & \text{if } X_{min } \leq X \leq X_{max}\\ \text{refuse prediction, e.g. NA} & \text{else} \end{cases} $

It is not sensible to formulate such a condition on $Y$. You model the $Y$ as the unknown. But you cannot base a decision on a property you do not know.

You could trace this condition on $Y$ back to certain $X$, but this corresponds again to posing a constraint on the known variates (which are all you have when evaluating the model). And both for your modeling and for judging the quality of your model, you need all data points that are inside the specification for the knowns ($X$), so trimming $Y$ would violate this condition.

Whether it is sensible to pose such a constraint on $X$ will depend on your application, there is nothing automatic about this.

Note also that I formulated the constraint with values $X_{min}$ and $X_{max}$, not with quantiles. This is important. You may deterimine these values by looking at the quantiles in the modeling process, but in the ready-to-use model, you need quantities of $X$ to compare against: the model must be usable for a single prediction, where you cannot compute a quantile.

However, you need to take into account not only the direction of your model $Y$ ~ $X$, i.e. $X$ predicts $Y$ but also the purpose of the model.

A restriction on certain ranges of $X$ is sensible for predictive models. E.g. in my field (chemometrics) this is rather emphasized. The default restriction is that a model should never be used outside the $X$ range covered during both modeling and validation. In other words, no extrapolation. But you may define more restricrive $X$ values. Note however, that we chemometrician can usually point out physical and/or chemical reasons why we can reasonably assume a e.g. linear relationship for intermediate $X$, but neither for too small, nor for too large $X$s. This is "automated" only in the sense that one of the first things to think about when setting up a chemometric regression is the range you need to cover for your application, and/or measurin the range you are able to cover.

Whether restricting the $X$ can be justified for a descriptive model is IMHO a more difficult question. In any case I'd at least report what the function does with the excluded $X$s, and how this compares to their $Y$s.

$\endgroup$
2
$\begingroup$

Regression is carried out conditional on the observed values of the independent variables. So if you only want to model over a certain range and you're not confident in the assumption of a linear relationship with the response (or another assumption such as constant variance) outside this range it can be reasonable to exclude observations with predictor values outside it - & the analysis will be valid just as if you'd fixed the dependent variables at those values in an experiment. (All the same, the practice of automatically excluding a fixed proportion of the observations with the most extreme predictor values seems hard to justify.)

What's wrong with 'trimming' the dependent variable is that you're introducing something into the data-generating process that you don't take into account in the analysis, & moreover losing the information, whether about the parameter or about fit, carried by those excluded observations. I'm not sure whether you meant excluding outliers in the dependent variable or in the residuals - whuber & Greg have described the problems with both practices.

$\endgroup$
1
$\begingroup$

I don't know about financial economics, but there are instances when our understanding is so embarrassingly inadequate that discarding data is the only thing we know how to do. For example, the field of energy intake/consumption and calories sometimes entails using data driven approaches to discard extreme observations that we know are not plausible.

If there are valid reasons to routinely discard data in financial economics (and with valid I mean that such data points are not plausible), surely there are better approaches then simply remove 0.5 or 1% of the data at either end. And if the data points are simply large/extreme but plausible, there are robust methods, transformations etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.