7
$\begingroup$

The alpha parameter of an exponential moving average defines the smoothing that the average applies to a time series. In a similar way, the window size of a moving window mean also defines the smoothing.

Is there some way to tune the alpha parameter such that the smoothing is approximately the same as that of a moving window mean of a given size? (Not looking for identical results, obviously, and offsets are OK). So, say tune alpha such that the resultant time series is as close as possible in shape to the result provided by a 3-month moving window?

edit: context: I'm trying to generate multiple proxy for soil moisture, from rainfall data, which abstractly represent different depths (which I'm assuming are related to long-term rainfall averages). A moving window allows me to calculate e.g total rainfall in the past 3 days, 3 months, or year, which might correspond to top few centimetres of soil, the top meter, and the extended soil column, respectively. However, a moving window requires data from the past, which isn't always available (e.g. at the start of a series). If an exponential average is used instead, then I only need to store one value for each average (the average from the previous time step), and this value can be initialised with the long-term mean.

$\endgroup$
  • $\begingroup$ What kind of offset are you referring to? $\endgroup$ – Glen_b -Reinstate Monica Jul 12 '16 at 3:13
  • $\begingroup$ @Glen_b: as in, if the peaks and troughs in the two means are shifted relative to each other. I don't have a reason to expect them to be, but I'm not sure that they wouldn't be, either. $\endgroup$ – naught101 Jul 12 '16 at 3:24
  • $\begingroup$ Oh, sorry, I had assumed you meant you wanted to find an alpha parameter that implied close-together fitted(/predicted) values (close in conditional mean). Do you intend instead an alpha parameter that implies a similar level of smoothing even though the values may be quite different? (in effect, something like being close in conditional variance rather than close in conditional mean) $\endgroup$ – Glen_b -Reinstate Monica Jul 12 '16 at 3:42
  • $\begingroup$ Oh, with your moving window mean, is that backward-looking or centered on the observation? (i.e. $\hat{y}_y=\frac{1}{k}\sum_{i=0}^{k-1} y_{t-i}$ vs $\hat{y}_y=\frac{1}{2k+1}\sum_{i=-k}^{k} y_{t-i}$) (keeping in mind that exponential smoothing is generally only looking backward) $\endgroup$ – Glen_b -Reinstate Monica Jul 12 '16 at 4:08
  • $\begingroup$ Added some context, just in case it might help clarify the intent of the question. However, now that I think about it, I'm wondering if an exponential moving average is even valid on (exponential) rainfall data... $\endgroup$ – naught101 Jul 13 '16 at 2:37
6
$\begingroup$

Let $x$ be the original time series and $x_m$ be the result of smoothing with a simple moving average with some window width. Let $f(x, \alpha)$ be a function that returns a smoothed version of $x$ using smoothing parameter $\alpha$.

Define a loss function $L$ that measures the dissimilarity between the windowed moving average and the exponential moving average. A simple choice would be the squared error:

$$L(\alpha) = \|x_m - f(x, \alpha)\|^2$$

If you want the error to be invariant to shift/scaling, you could define $L$ to be something like the negative of the peak height of the normalized cross correlation.

Find the value of $\alpha$ that minimizes $L$:

$$\underset{\alpha}{\min} L(\alpha)$$

Here's an example using a noisy sinusoidal signal and the mean squared error as the loss function:

enter image description here

Another example using white noise as the signal:

enter image description here

The loss function appears to be well behaved and have a single global minimum for these two different signals, suggesting a standard 1d optimization solver could work (as I used to select $\alpha$ here). But, I haven't verified that this must be the case. If in doubt, plot the loss function and use a more sophisticated optimization method if necessary.

Edit:

Here's a plot of the optimal alpha (for exponential smoothing) as a function of window size (for simple moving average). Plotted for each of the signals shown above.

enter image description here

$\endgroup$
  • $\begingroup$ Hrmm. So, this is a fair enough approach, but it kind of assumes that the alpha will be different for each time series, for a given window size. Is that necessarily the case? I was thinking that there might be some general analytic solution... $\endgroup$ – naught101 Jul 12 '16 at 3:27
  • $\begingroup$ I hoped so too. This is just an empirical argument, but I tried for some different signals and the value of alpha could be different, even when window width was the same. $\endgroup$ – user20160 Jul 12 '16 at 3:34
  • $\begingroup$ What is the window you used for these plots? Pandas' exponential moving window has a "centre of mass (com)" argument that is defined as: $\alpha = 1 / (1 + com)$, which seems to roughly match the window length in a normal moving window. Do your plots also have this relationship? $\endgroup$ – naught101 Jul 14 '16 at 5:19
  • 1
    $\begingroup$ I edited the post to show the optimal alpha as a function of window size. The function is indeed smooth, but can can differ depending on the signal. The optimization took only ~1ms for 1000 samples and ~160ms for 1e6 samples, so it's not all that burdensome. But if you want to avoid it, you could generate an alpha vs. window size curve for a single, prototype signal, then use it to choose alpha for further signals. $\endgroup$ – user20160 Jul 18 '16 at 17:15
  • 1
    $\begingroup$ Alternatively, you might be able to derive a closed for expression for a simple signal w/ well-defined statistics, then just accept any small innacuracies that arise from differences between the model signal and your actual signals. $\endgroup$ – user20160 Jul 18 '16 at 17:15
2
$\begingroup$

If I understand the question correctly, the issue is one of trying to make an exponentially decreasing weight series fit to a discrete uniform (constant weight with cutoff):

![Comparison of EWMA weight functions with ordinary moving average

Clearly either an EWMA decreases quickly (fitting badly at older lags where the ordinary moving average still has high weight) or has a tail much further into the past, fitting the weight-distribution badly where the ordinary moving average has no weight).

Exactly which choice of $\alpha$ will do best at matching the results from uniform-weights will depend crucially on how you measure performance and (naturally) on the characteristics of the series (both ordinary moving average and EWMA would only be reasonably suitable for weakly-stationary series for example, but that covers a lot of cases with potentially different relative performance for different $\alpha$ values)

The question leaves both of these things vague, so I suspect there's not a lot more to be said than "it depends" - about either the similarity of the conditional mean or the size of the conditional variance, here.

$\endgroup$
0
$\begingroup$

We can think of this as a hyperparameter optimization problem.

We have a target X_mean which is the target value.

We also have a loss function e.g. L2 (X_exponential - X_mean).

We are searching for a hyperparameter (alpha) for the exponential moving average to minimize loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.