You can downdate a variance in exactly the same fashion as you update it; you "undo" the operations that were required to add the observation in the first place (such as subtracting from totals instead of adding).
Take the observation you want to remove. Imagine you just added that observation as the $n$-th observation (of $n$), using one of the algorithms you point to. Now undo those steps you just did, to take it back to the way it was just before it was added (the variance as at $n-1$ observations).
However, an important caveat: updating (done properly) tends to be quite stable, downdating tends to be less so. You can deal with removing the occasional observation (under some restrictions) but if you're doing a lot of adding and removing, loss of numerical accuracy tends to be more of a problem. (Removing largest values may tend to be worse, so additional caution may be required.)
If you have a specific application in mind (such as a windowed standard deviation), it would be best to mention it explicitly.