Is it possible to forget data when computing sufficient statistics on a stream?

My understanding is when we are computing sufficient statistics on a stream, when a new instance arrives, the value of the new instance is added to the already computed sufficient stats. so there is no way of forgetting/decrementing any data since the value of all the current data is summarised in sufficient stats but this is just an intuitive explanation. how do I proof this using the formula?

Thanks

• "Added to" is, unless meant very loosely, not general enough: the sufficient statistic might be the sample maximum, say. – Scortchi - Reinstate Monica Jun 13 '17 at 8:59
• hi, the question is if we defined the concept of sufficient statistics assuming that we add observations with the ”Increment” operator. Can the sufficient statistics be computed if we also forget data with the ”Decrement” operator? – MugB Jun 13 '17 at 9:02
• @MugB aren't you in effect restricting yourself to the exponential family? In that family, the sufficient statistic of a set of independent identically distributed data observations is simply the sum of individual sufficient statistics – Glen_b Jun 13 '17 at 10:52
• @Glen_b if I understand you correctly, what you are saying is say if at timepoint 1, I have 100 data points for which i compute the sufficient stats for and at time point 2 I see another 100 data points and compute the sufficient stats again and say i am now considering that the data points in t1 is old and I want to forget them (considering limited memory, computational complexity, etc) so does that mean that I don't actually have to use the decrement operator? – MugB Jun 13 '17 at 17:31

Simply write the value out. Here for the mean: $$\mu_n = n^{-1}\sum_i^n{x_i}=n^{-1}(\sum^{n-1}_i{x_i}+x_n)=n^{-1}((n-1)\mu_{n-1}+x_n)$$ And it is proved. For variance check out https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance online algorithm. In order to include forgetting to he variance set n = min(n,1/gamma), where gamma is the forgetting factor.
• Use a forgetting factor: $$\mu_n =(1- \gamma_n)*\mu_{n-1}+\gamma_n*x_n$$ Where $$\gamma_n=max(1/n,\gamma)$$ So here you are forgetting with a factor of gamma when 1/n becomes smaller than gamma. So if you have 100 observations and a gamma of 0.05 then the newest observation have a weight of 1/20 and therefore have a higher weight than the past observations. Which are gradually being forgotten. – Peter Mølgaard Pallesen Jun 13 '17 at 9:12