0
$\begingroup$

For a given variable $X$, we compute the standard deviation. Now I removed $k$ observations from $X_n$ and I would like to compute the new standard deviation $\sigma_{(k)}$ using $\sigma_{n}$.

I found some algorithms to compute the standard deviation on-line for without one observation like this [formula][2]:

$\sigma_n^2=\frac{n-2}{n-1}\sigma^2_{n-1}+\frac{1}{n}(X_n-\bar{X_{n-1}})^2$

But it does not adapt to removing $k$ observations at once.

1- Is there a way to do it for $k$ observations ? (It works for the mean)

2- For which $k$ this is less expensive than computing the standard deviation from scratch?

$\endgroup$
1
  • 1
    $\begingroup$ I think your (2) depends more on $n$ than on $k.$ $\endgroup$
    – BruceET
    Commented May 24, 2020 at 10:26

1 Answer 1

0
$\begingroup$

Comment: Won't fit in comment box.

Use $$S^2 = \frac{1}{n-1}\left[\sum_{i=1}^n X_i^2 - \frac{1}{n}\left(\sum_{i=1}^n X_i\right)^2\right]\\ = \frac{1}{n-1}[Q -T^2/n].$$

Quantities $Q$ and $T$ can be found from $S^2$ and $\bar X.$ Then they can be adjusted to account for removed observations.

Finally, use adjusted $Q^\prime$ and $T^\prime$ to compute the new $S^2.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.