Removing Variance in Time Series After Applying Log Transformation

Question

I'm trying to look at natural gas prices from 2003-2018. The issue is after applying log transformation and then diffrencing data by 1, I still seem to get an increase in variance from mid 2014-2018. Do I need to perform another transformation to keep progress stationary?

energy['Midpoint']=energy['Midpoint'].mask(energy['Midpoint'].sub(energy['Midpoint'].mean()).div(energy['Midpoint'].std()).abs().gt(2))
energy_transformed=energy.copy()
energy_transformed['Midpoint']=np.log(energy_transformed.iloc[:]).diff(1)

Answer 1

Five (not mutually exclusive) possibilities come to mind:

1. The Slutsky http://www-history.mcs.st-andrews.ac.uk/Biographies/Slutsky.html Effect ... Unnecesaary differencing can INJECT variability. Consider the variance of a random process that is differenced OR unnecessarily filtered http://mathworld.wolfram.com/Slutzky-YuleEffect.html

2. Variance changes can be determinstic i.e. variance changes at particular points in time remediable by GLS ...see http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html

3. One time Pulses or anomalies that are untreated.

4. Changes in an intervention effect that has not been accounted for.

5. Changes in the underlying ARIMA (stochastic) model/paramaters over time .

In general take a look at http://stats.stackexchange.com/questions/18844/when-and-why-to-take-the-log-of-a-distribution-of-numbers

In terms of removing variability I would form an ARMAX model that included possible level shifts , local time trends, pulses and seasonal pulses while identifying the need for a power transform or weighted regression structure . The idea here is to transform when the error variance from a useful model is categorized as heterogenous.

Finally you might want to peruse Variance inhomogeneity in time series when forecasting suggesting that there may be different variability for different months.