17
$\begingroup$

I wrote a simple function in Python to calculate the exponentially weighted mean:

def test():
  x = [1,2,3,4,5]
  alpha = 0.98
  s_old = x[0]

  for i in range(1, len(x)):
    s = alpha * x[i] + (1- alpha) * s_old
    s_old = s

  return s

However, how can I calculate the corresponding SD?

Improve this question
$\endgroup$
3
  • $\begingroup$ Are you after the standard error of the mean, or some estimate of the standard deviation of the process? $\endgroup$
    – Glen_b
    Commented Aug 14, 2014 at 13:56
  • $\begingroup$ @Glen_b I am trying to use this to see how much a stock price deviates from the exponentially-weighted mean by some multiple of the "standard deviation". Which one would you recommend? $\endgroup$
    – Mariska
    Commented Aug 14, 2014 at 14:07
  • 1
    $\begingroup$ From what I can see, there's a fundamental conflict (or inconsistency) underlying this question. People use the EWM when they do not care to analyze the data to characterize and quantify the serial correlation, but in order to answer this question the serial correlation must be estimated; but then why would you use the EWM in the first place? $\endgroup$
    – whuber ♦
    Commented Aug 14, 2014 at 14:37

1 Answer 1

Reset to default
24
$\begingroup$

You can use the following recurrent formula:

$\sigma_i^2 = S_i = (1 - \alpha) (S_{i-1} + \alpha (x_i - \mu_{i-1})^2)$

Here $x_i$ is your observation in the $i$-th step, $\mu_{i-1}$ is the estimated EWM, and $S_{i-1}$ is the previous estimate of the variance. See Section 9 here for the proof and pseudo-code.

Improve this answer
$\endgroup$
6
  • $\begingroup$ using the above formula and the list [1,2,3,4,5], I got SD = 0.144, whereas the normal Sample SD is 1.58. There is a factor of 10x between the two different SD's. Is this normal? $\endgroup$
    – Mariska
    Commented Aug 15, 2014 at 2:12
  • 5
    $\begingroup$ Using $\alpha = 0.98$ you also get the mean = 4.98, which is equally useless. :) Using such coefficient, you put almost all weight on the last measurement. More realistic values of $\alpha$ are close to zero, in that case they account for long-range average. For your example, try $\alpha = 0.2$, but in practice you will probably need to average more measurements, so the values around $\alpha = 0.01$ are more realistic. $\endgroup$
    – Roman Shapovalov
    Commented Aug 15, 2014 at 7:30
  • 2
    $\begingroup$ The Section 9 link is broken, and the shape of the formula looks odd to me. I was expecting sth like (1-a)*sth + a*sth, instead of (1-a)*(...). $\endgroup$
    – Albert Netymk
    Commented Aug 19, 2020 at 15:18
  • 1
    $\begingroup$ +1 is there an updated link for the Section 9? What's the intuition behind the formula being shaped like that instead of how @AlbertNetymk suggested. $\endgroup$
    – Dylan Kerler
    Commented Jan 8, 2022 at 12:25
  • 4
    $\begingroup$ The paper is available at archive.org here. For future reference, the paper's title is "Incremental calculation of weighted mean and variance" written by Tony Finch, Feb 2009 (in case this link also gets broken). $\endgroup$
    – Rok Povsic
    Commented Feb 3, 2022 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.