1
$\begingroup$

I have a situation that when I fit a robust regression line (least trimmed squares) to a set of data a lot of the residuals are in fact zero.

This occurs mainly in the situation where the slope is zero and the y values are integers. When the line is fitted it runs right through the majority of the values. I am happy with this fit as a traditional least squares line is incorrect due to a few errors in the data.

However now I've fitted the line I want to analyze the residuals and detect the outliers (possibly automatically). I intended to compute a 'score' perhaps based on Tukey's technique of using the upper and lower quartile plus 1.5 times the inter-quartile range. However this approach won't work if many of the residuals are zero because the IQR is also zero.

What should I do? Just base the score on something like the number of standard deviations from the mean? Thanks.

Improve this question
$\endgroup$
9
  • $\begingroup$ What about the X values, are they all continuous? $\endgroup$
    – user603
    Jul 30, 2013 at 8:26
  • $\begingroup$ @user603. The data are actually time series data (typically monthly), so I am exploring a couple of different approaches. $\endgroup$
    – Antonio2100
    Jul 30, 2013 at 9:00
  • $\begingroup$ Is your data stationary? (e.g. first difference, or trend adjusted?) $\endgroup$
    – user603
    Jul 30, 2013 at 9:13
  • $\begingroup$ @user603 No I haven't applied any differencing. In the data above 'time' is the independent variable. There are still other issues I'm exploring with the data, but I was curious about whether other people have encountered this situation also. $\endgroup$
    – Antonio2100
    Jul 30, 2013 at 9:38
  • 1
    $\begingroup$ If your data is non stationnary, you might want to use the LTS-filter: it's the same approach as LTS, but as a filter. You will find more detail on page 23 of this document. $\endgroup$
    – user603
    Jul 30, 2013 at 9:42

2 Answers 2

Reset to default
1
$\begingroup$

Various thoughts:

  1. You say that problem is mostly when the slope is zero. But whenever the regression is flat, the problem is the same as that of flagging univariate outliers for the response variable.

  2. More generally, a sensible criterion may depend on the particular kind of robust regression you use, which you have now named as least trimmed squares. (There are many flavours: few seem to sustain any popularity for more than a few years except for the oldest, L1 or more generally quantile regression.)

  3. Why not just use the values of the residuals and plot them? Converting to residual/scale of residuals isn't always needed, even when you are using different response variables.

  4. Much depends on quite why and how much you want or need to automate. If you are doing this hundreds, thousands… of times, then understood. If only a few times, you can waste more time worrying how best to do it than just looking at some plots. It's the transition between those situations that's tricky.

Improve this answer
$\endgroup$
11
  • $\begingroup$ " few seem to sustain any popularity for more than a few years except for the oldest" which ones are you referring to? $\endgroup$
    – user603
    Jul 30, 2013 at 8:44
  • $\begingroup$ The pattern of robust regression research seems to be that researchers favour their own methods, disparage others and nothing remains top of anyone's list for long. I've seen e.g. biweight and least median of squares pushed hard by proponents and then criticised strongly as often poor. I've seen more recent methods pushed hard only to learn that they are painfully slow or impracticable for large datasets. But I'm emphatically no expert and would welcome counterexamples of good methods that have lasted a decade in popularity and are recommended in all monographs on robust statistics. $\endgroup$
    – Nick Cox
    Jul 30, 2013 at 8:51
  • $\begingroup$ @Nick Cox Thanks for your reply. I added to my question that I used least trimmed squares. Yes plotting the residuals does work well, except I have potentially thousands of sets of data to work with that is why I hope to find an automated solution. In some cases the regression is flat, but not in all or even most cases. I wonder if I should look at using different methods depending on whether the regression is flat or not. $\endgroup$
    – Antonio2100
    Jul 30, 2013 at 8:55
  • $\begingroup$ I think you need advice from someone very familiar with least trimmed squares. $\endgroup$
    – Nick Cox
    Jul 30, 2013 at 9:08
  • 1
    $\begingroup$ I was stating an impression, and do not have an expert opinion, as I said, and I am happy that you want to qualify or indeed to correct a throw-away remark, which is how this started. So, thanks for your remarks. Incidentally, least squares was introduced in 1805 or earlier, depending on whether you believe Gauss's priority claim, does predate MAD, which was 1816 if I recall correctly. I'd regard Boscovich as relevant here too. $\endgroup$
    – Nick Cox
    Jul 30, 2013 at 9:46
1
$\begingroup$

I had a similar problem where many of the residuals were zero and the IQR would not be usable because it was either zero or close to zero. A practical solution I found worked reasonably was to use the interdecile range instead of the IQR. I then used the upper decile and lower decile and made the outer fences a multiple of the IDR away from these.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.