How is the decay rate in exponential smoothing optimized?

Question

For the sake of simplicity, I just want to focus on single/level exponential smoothing. When alpha, the decay rate, is near 1, the most recent observation has the highest weight and influence of recent observations decay rapidly, lending to a high variance model. I'm curious how alpha can be optimized, returning a smooth approximation that reasonably closely follows the observed data.

I can see two perspectives:

What seems to be working for me is minimizing the sum of mean squared error and mean volatility (mean change between observations.) Though, I'm not sure that this is principled.

Regardless, how is alpha optimized in practice?

Answer 1

If your previous smoothed number was $y_{n-1}$ and your new observation is $x_n$

then the new smoothed number $y_n$ which minimises $\alpha(x_n-y_n)^2+(1-\alpha)(y_n-y_{n-1})^2$

is $y_n = \alpha x_n + (1-\alpha)y_{n-1}$

which is essentially exponential smoothing.

In your particular suggestion of equal weighting for $(x_n-y_n)^2$ and for $(y_n-y_{n-1})^2$ in your minimisation, you get $\alpha=\frac12$. You can make $\alpha$ larger if you want to be closer to the new number and smaller if you prefer to be closer to the previous smoothed number, but you should always have $0 \lt \alpha \lt 1$. Choosing $\alpha$ should reflect your preference for the balance between these two; it may be affected by your perception of the data and you might choose a higher $\alpha$ when you think the trend is stronger than the noise, and a smaller $\alpha$ when you think the noise is excessive.

Answer 2

Unfortunately, without a specific case in mind, it's a bit hard to answer! Reason simply being that as with many things, the true answer is "it depends".

I'll outline a couple of potential examples/uses/perspectives for you to consider. I will assume some kind of time series data that you're smoothing.

Scenario 1: You having some kind of series with trends but lots of noise around the trend, and you want to just get a better visual/intuitive understanding of what's going on, what the general trend or pattern is. In such a case, I would recommend taking some small portion of the data (first x points) and simply manually playing around with the value of alpha until visually, your smoothed line seems to nicely reflect the denoised/smoothed trend you want to see. Then, once you've arrived at the alpha that seems to work for that first set of the data, apply it on the rest.

Scenario 2: You want to use the smoothed data predictively in some kind of model (regression, machine learning etc). Here, I would apply a wide variety of alpha's (maybe 0.1, 0.2, 0.3...0.8, 0.9) and running your model on all of those as variables/features. Then, using cross validation and some basic feature selection, you would be able to select which alpha weighting(s) is/are most appropriate for the predictive task.

Scenario 3/other: Broadly speaking, if there is a quantifiable objective you're trying to achieve (find the alpha that best approximates ...), then I think the way to go is just using some kind of cross-validation to try out different alphas and apply it on unseen data.

This is all very general (as is the question), and if you have more specific questions or if I was unclear, feel free to ask!