How is the decay rate in exponential smoothing optimized?

For the sake of simplicity, I just want to focus on single/level exponential smoothing. When alpha, the decay rate, is near 1, the most recent observation has the highest weight and influence of recent observations decay rapidly, lending to a high variance model. I'm curious how alpha can be optimized, returning a smooth approximation that reasonably closely follows the observed data.

I can see two perspectives:

What seems to be working for me is minimizing the sum of mean squared error and mean volatility (mean change between observations.) Though, I'm not sure that this is principled.

Regardless, how is alpha optimized in practice?

• I'd love to answer the question, but could you first provide a bit more detail about the context of the problem? For example, what are you trying to achieve with this optimization? What is the objective function for it (like, you're trying to optimize with respect to what?). What type of time series data are you working with? It's hard to answer the question precisely without more info, especially to the first question. Are you doing time series prediction using moving averages, and you're trying to figure out what level of smoothing is best? Jan 19, 2022 at 23:07
• Hi @VladimirBelik, I'm just reading about exponential smoothing; at this point in time, it's purely educational w/ no real source data and/or problem. Jan 20, 2022 at 0:08
• Why care about mean volatility? Is that supposed to encode some prior knowledge about the process? I believe the common ways of optimization pay no attention to that, only to fit. Jan 20, 2022 at 6:49

If your previous smoothed number was $$y_{n-1}$$ and your new observation is $$x_n$$

then the new smoothed number $$y_n$$ which minimises $$\alpha(x_n-y_n)^2+(1-\alpha)(y_n-y_{n-1})^2$$

is $$y_n = \alpha x_n + (1-\alpha)y_{n-1}$$

which is essentially exponential smoothing.

In your particular suggestion of equal weighting for $$(x_n-y_n)^2$$ and for $$(y_n-y_{n-1})^2$$ in your minimisation, you get $$\alpha=\frac12$$. You can make $$\alpha$$ larger if you want to be closer to the new number and smaller if you prefer to be closer to the previous smoothed number, but you should always have $$0 \lt \alpha \lt 1$$. Choosing $$\alpha$$ should reflect your preference for the balance between these two; it may be affected by your perception of the data and you might choose a higher $$\alpha$$ when you think the trend is stronger than the noise, and a smaller $$\alpha$$ when you think the noise is excessive.

• Also, to estimate alpha, you can run a random walk + noise kalman filter and then take the parameters estimated from that and convert to the equivalent $\alpha$ on SES. Check out Andrew Harvey's text for the gory details of converting back to exponential smoothing. amazon.com/Forecasting-Structural-Models-Kalman-Filter/dp/… Jan 20, 2022 at 0:30

Unfortunately, without a specific case in mind, it's a bit hard to answer! Reason simply being that as with many things, the true answer is "it depends".

I'll outline a couple of potential examples/uses/perspectives for you to consider. I will assume some kind of time series data that you're smoothing.

Scenario 1: You having some kind of series with trends but lots of noise around the trend, and you want to just get a better visual/intuitive understanding of what's going on, what the general trend or pattern is. In such a case, I would recommend taking some small portion of the data (first x points) and simply manually playing around with the value of alpha until visually, your smoothed line seems to nicely reflect the denoised/smoothed trend you want to see. Then, once you've arrived at the alpha that seems to work for that first set of the data, apply it on the rest.

Scenario 2: You want to use the smoothed data predictively in some kind of model (regression, machine learning etc). Here, I would apply a wide variety of alpha's (maybe 0.1, 0.2, 0.3...0.8, 0.9) and running your model on all of those as variables/features. Then, using cross validation and some basic feature selection, you would be able to select which alpha weighting(s) is/are most appropriate for the predictive task.

Scenario 3/other: Broadly speaking, if there is a quantifiable objective you're trying to achieve (find the alpha that best approximates ...), then I think the way to go is just using some kind of cross-validation to try out different alphas and apply it on unseen data.

This is all very general (as is the question), and if you have more specific questions or if I was unclear, feel free to ask!