1
$\begingroup$

I'm working with a data source that occasionally includes missing values. I noticed that this caused tso() to miss obvious outliers even though they are nowhere near the missing value- for example:

library(tsoutliers)

series <- c(0,-0.025,0.005,0.03,0.078,0.042,0.034,0.04,0.04,0.019,-0.005,0.057,
                0.065,0.247,0.87,-0.03,-0.006,0.008,0.009,0.022,-0.005,0.05,0.015,
                0.018,0.025,0.058,0.012,0.076,0.078,3.31,-0.042,0.016,0.019,0.037,
                0.045,0.08,0.03,0.057,0.095,0.044,0.132,0.044,0.088,0.198,0.314,
                0.063,0.037,0.008,0.028,0.041,0.073,0.069,0.073,0.059,0.037,0.05,
                0.006,0.081,0.064,0.373,0.084,0.028,0.053,0.042,0.04,0.083,0.01,
                0.047,0.057,0.05,0.111,0.054,0.115,0.114,0.243,0.26,0.018,0.043,
                0.049,0.062,0.127,0.055,0.065,0.035,0.018,0.068,0.019,0.138,0.364,
                0.87,-0.04,-0.009,0.026,0.023,0.065,0.079,0.106,0.079,0.037,0.102,
                0.09,0.03,0.322,0,0,-0.022,0.026,0.065,0.028,0.072,-0.038,0.01,
                0.048,0.047,0.088,0.013,0,NA,0.051,0.012,0.029,0.028,0.01,0.033,
                0.021,0.031,0.036,0.106,0.057,0.028,0.125,0.136,0.415,0,-0.009,
                0.023,0.036,0.005,0.022,0.03,0.078,0.026,0.099,0.133,0.062,0.145,
                0.187,0.02,0.087,0.002,0.07,0.054,0.089,0.063,0.071,0.065,0.062,
                0.157,0.175,0.076,0.163,0.308,0.51,0.136,0.011,0.055,0.046,0.053,
                0.035,0.038,0.04,0.039,0.073,0.055,0.007,0.036,0.192,0.393,-0.039,
                -0.013,0.042,0.037,0.058,0.094,0.092,0.102,0.061,0.086,0.144,0.057,
                0.079,0.42,0.99,-0.037,0.021,0.031,0.071,0.022,0.055,0.06,0.082)
s <- ts(series)

plot(tso(s, remove.method = "bottom-up"))

Below you can see the plot- notice only one of the spikes was captured, and the biggest one by far was ignored. Not shown here is the same graph but with the "NA" simply removed from the series before sending it to the tso() function- it looks more intuitive, so the NA is likely the problem.

tso output

Is there a methodological reason for this or is it a software issue? And is there a good workaround? Thank you! (this is tsoutliers v 0.6.5, R 3.2.3)

$\endgroup$
1
$\begingroup$

It's a software issue. I found that the presence of NAs propagated along the t-statistics and prevented the detection of outliers at some points. I've worked around this issue by replacing NAs in the residuals with the mean of the remaining residuals. (The updated version of the package is available on CRAN.)

This is what the function returns now (note that the argument remove.method is now deprecated, discard.method should be used).

require("tsoutliers")
plot(tso(s, discard.method="bottom-up"))

detected outliers

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.