2
$\begingroup$

This has been driving me crazy, so I thought I'd come and ask for some help:

I'm doing a study involving content analysis of newspapers. The unit of analysis is each day, adding all stories for the day. Overall, there are 4 variables for each case (i.e. number of stories per day, column-inches total, etc.). After collecting data for a while, I did notice some of the variables in some of the cases were quite different (higher, mostly) than the others, and started considering outliers. Following the basic concept of 3 std. dev., I started checking if some variables did show outliers - and they did.

These outliers are, mostly, days when something unusual happened (a big, breaking story that mattered to that audience of that paper; it's not an error, but an anomaly, I'd say).

So, here's the question:

I do know there's tons of discussions of removing or not outliers from analysis, and don't think there'll ever be an answer for it plain and simple. But, if I DO decide to remove the outliers:

a) Should I remove them before or after describing the frequencies? i.e. "During the X days analyzed, there were 765 stories" - should the 765 include the outliers or not? Should I remove outliers only when comparing means and significant correlations, or remove them altogether?

b) If I do remove the outliers, since they only refer to 1 (2 in very rare cases) of 4 variables I'm collecting for each case, should I remove the whole case altogether or just the outlier variable (leading to different N's in the end for each variable)?

Input on this will be extremely helpful. Thank you SO much in advance!! D.

$\endgroup$
  • $\begingroup$ Do you know how to use R? From there, it'll be a minute. $\endgroup$ – user603 Jul 24 '12 at 15:52
1
$\begingroup$

If the number of stories per day is systematic with the day of the week , or the week of the year , or a particular day in the month , or if there has been shifts in the mean over time or changes in trends over time , or if holidays have a persistant effect then one needs to control for any of these statistically significant factors BEFORE one can think of using some +/- criterion. In addition if there is memory i.e. some form of auto-regressive behavior in the series this must be accounted for in order to establish anomalies. Note that one can't simply look at the acf/pacf of the original series in order to determine seasonality if some of these factors come into play. Pursue Transfer Function Identification integrated with Intervention Detection.

$\endgroup$
1
$\begingroup$

Using three sigma limits is not the way to search for outliers. IrishStat and I have spoken on this numbers of times on CV with respect to time series and in my case the use of influence functions. You can find my AJMMS 1982 paper "The Influence Function and its Application to Data Validation, Gnanadesikan's multivariate analysis book, the work of Ruey Tsay on Time Series and the major papers and book by Doug Martin and his colleagues. Just search CV for the key word "outliers" and "influence functions" and you will ifnd tons of material.

Keep in mind outlier detection should not be used as an automatic data removal procedure. After you find them figure out why they are there.

$\endgroup$
  • $\begingroup$ Thank you so much for the replies, everyone. Michael, I absolutely agree with your statement of "outlier detection should not be used as an automatic data removal procedure," and that's what I'm trying to figure out right now. Since I'm dealing with content analysis of news stories, outliers are "special news" (i.e. NBA finale), where the numbers for that ONE day are VERY different from other days, and I'm worried would skew any analysis (i.e. mean per day of the week). So, I know the why, but not about the "so..." based on the two Qs I posed (or whether to just ignore them being outliers). $\endgroup$ – Danny Jul 24 '12 at 17:43
  • $\begingroup$ Just for clarity, Michael and I are in total agreement except we disagree on some things (grin !). We definitely agree on attempting to find out WHY the unusual values occurred and then incorporating/converting that knowledge into valid predictor series that had been inadvertently omitted. What we disagree on is that I would include identified dummy variables whether or not I knew what caused them because with them I have a robust estimator of all the other coefficients and without them I have chaos. $\endgroup$ – IrishStat Jul 24 '12 at 20:14
  • $\begingroup$ @IrishStat Actually I think we are close to agreeing on that. I am a big believer in identifying outliers on the basis of there influence on estimates and predictions. I think your comment about using dummy variables to reduce their "bad" influence on future predictions is legitimate. It is also in line with the philosophy of Peter Huber and Frank Hampel who advocate robust statistical methods that downweight outliers rather than ignore and eliminate them. As a practtical matter if you can't find the reason for the outlier I think I agree with what they do and what you do. $\endgroup$ – Michael R. Chernick Jul 24 '12 at 21:01
  • $\begingroup$ @IrishStat continued. But I think we can agree that it is far better if you can determine the cause of the outlier. Then the option to delete may be justified or on the other hand you may have learned about a phenomena that can be predicted by a covariate to reoccur at certain points in time. $\endgroup$ – Michael R. Chernick Jul 24 '12 at 21:04
  • $\begingroup$ Thank you again. This being a content analysis, it's simple to identify the reason for the outliers - in this case, special events (news) that leads to increased coverage. So the reason for their appearance is actually very simple; the question remains, however, what to do about them so those (overwhelming) events don't skew the whole sample - or should they? (Please note that the cases are independent, so one case having an outlier doesn't mean another one having a similar one, at the same time or at any time). Thanks again! $\endgroup$ – Danny Jul 26 '12 at 4:05
1
$\begingroup$

You've gotten some good answers already, and I don't know time-series analysis well, so let me just make a couple of notes.

  1. I'm pretty sure your data aren't / can't be normally distributed (it looks like they're counts) so these may not be outliers--such distributions by nature are often highly skewed.
  2. I can't tell if the variables in question are explanatory variables or response variables. Calling something an 'outlier' makes less sense for for an explanatory variable, although such a point can have very high influence over the betas you estimate. This may be a good thing. One way to get more information about how much a given point is driving your results is to fit your model both with and without the point in question and see how that beta changes (a measure called dfbeta).
  3. Should these points be outliers, you want to use a different / robust loss function. Note that just throwing out the outliers amounts to a different loss function, just one with less desirable long run mathematical properties. In standard regression contexts, perhaps the most common robust loss function is Tukey's bisquare, but I don't know if it's used with time-series.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.