# Transforming time series to compensate for change in variance

I have a time series (shown below) that comes from a sensor whose calibration was changed in the middle of last year. As part of this change, the sensor's reading of the variance (or volatility) of the observations increased, although actually, this volatility did occur before the change in calibration, but it was not being fully detected.

As the data the sensor detects is highly seasonal (at both the weekly and yearly level), I want to create an accurate forecast of what the future readings will be, using the new level of calibration. I don't want to throw out the data from before the calibration, as we have only collected a little more than a year's worth of data.

I've tried traditional transformations (like Box-Cox) and a seasonal decomposition using stl() in R's forecast package, but the transformation does not solve this variance issue, as there's different variances at the same level of the data (say at 0.0, for example).

Are there any other techniques are there (that are available in the R programming language) that allow me to transform the data from before mid-2013 to match the variance of the data that comes later?

• What does seasonal at the yearly level mean? Would equivalently say that there are week and year fixed effects? That there is a a long and short memory process going on? – BKay Sep 4 '14 at 18:39
• Post your data in an excel form and I will demonstrate what you have to do . – IrishStat Dec 27 '14 at 12:24

Instead of normalizing, why don't you model the change in variance explicitly? Create a covariate that is equal to 0 before mid-2013 and to 1 after, and use it in the specification of the variance component.

• I'm trying to extract the seasonal component on this time series -- to my knowledge, stl() does not incorporate covariates. Is there another method I can use in R? – Bryan Aug 28 '14 at 1:01
• That I don't know. However, the problem may be that apart from the unconditional variance, other parameters may have changed after calibration. If you want to analyse the entire dataset, then you must think something remained unchanged. What is it? – James Aug 28 '14 at 14:15
• Yes, I'm trying to extract the yearly seasonality form the data -- so that's the parameter that's unchanged. – Bryan Aug 28 '14 at 16:46

Here's the solution I ginned up. The function expects to receive stationary time series, so before I put the two time series in question in the function, I differenced them.

equalizeVariance <- function(targetSeries, referenceSeries){
referenceSD <- sd(referenceSeries, na.rm=TRUE)
targetSD <- sd(targetSeries, na.rm=TRUE)
existingSDdifference  <- referenceSD - targetSD
optimalDivisor  <-  1
#lnFirstSeries<-BoxCox(firstSeries, lambda)
for (i in seq(0.01, 50, .001)) {
transformedTargetSeries <- targetSeries/i
transformedSD <- sd(transformedTargetSeries, na.rm=TRUE)
transformedSDdifference <- referenceSD - transformedSD
if (transformedSDdifference>existingSDdifference & transformedSDdifference<=0){
optimalDivisor <-  i
}
else {
print(paste(optimalDivisor, " is the optimal divisor", sep=""))
break
}
}
returnedSeries <- targetSeries/optimalDivisor
return(returnedSeries)
}


A solution for this problem is available in AUTOBOX , an advanced commercial program dedicated to time series analysis. A model is developed which may or may not include intervention variables to deal with level shifts , seasonal dummies and/or local time trends. A test is conducted ala Tsay http://www.unc.edu/~jbhill/tsay.pdf to automatically test for error variance deterministic change leading to a Generalized Least Squares solution. This approach is also called Weighted Least Squares.