# Clearing out errors from a data set

Sorry for the vagueness of the title, I am having a hard time even coming up with sort of problem I am facing (if there is a specific name for it....)

In a nutshell, I have a time series of points, something of this sort: Y1,0,0,Y4,0,Y6,0,Y8,0,0,... (this would be the y-axis of the graph, each point represents a time T, and each point is equally spaced out in terms of time.

The Problem: The 0 values denote a time were a cycle has been missed. The Y values denote the Y value at that time, and possibly, also previous value(s) that were missed before, meaning that, Y4 could potentially contain the actual value of Y2 and Y3 (other than its value), something like so: 5,0,0,15,0,5,0,15,0,0. In reality, the actual Y values would be something of the sort: 5,5,5,5,5,5,5,5,0,0. So basically, some of the values are deflated (0), and the others, are inflated (Y). (In a real case scenario the Y values would fluctuate, forming peaks and troughs).

So basically I have two types of values:

• Values which are deflated (represented by a 0).
• Values which are inflated (represented by anything greater than 0).

The issue is that these values always fluctuate. I was thinking of applying something similar to a rolling average, but that approach appeared too naive for my taste.

Is there something I can look into?

• are you implying that a non-zero after a zero needs to be apportioned to the 0 's in the past since the last non-zero ? I am a little bit confused ..it might be easier to talk (bilateral communication) rather than back and forth messages. – IrishStat Mar 20 at 0:04
• @IrishStat: Basically any point that we have is not accurate, since a 0 is a missed cycle, and a non 0 contains, potentially, more than 1 value. Since there is per second information, what I was planning on doing was to take 1 minute samples (each 1 minute point would be the average of 60 points) and then generate other points between P1 and P2 (where P1-P2 is 60s) that are a certain standard deviation away.So if P1 is less than P2, then the points will start getting a larger Y value as I move to the right. – npinti Mar 21 at 7:34