5
$\begingroup$

(As a caveat, I think this belongs on this stack site, but I'm not 100% sure.)

We have a time series that is physically sampled with only 8bit resolution, so we wind up with a "staircase" pattern, which in itself isn't an issue. The problem however, is that with any noise this flip-flops between the 2 states (as seen in the figure), and we really need a smoother signal.

I was wondering if there was any "standard" approach of doing this, or if anyone had any ideas. Currently we're using a Gaussian convolution filter, but that tends to insert a phase shift in the signal.

Thanks,

Aaron

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ Is the 8-bit value recording a truncated, or a rounded version of its input (or indeed, something else)? $\endgroup$
    – Glen_b
    Commented Apr 8, 2016 at 1:22
  • $\begingroup$ It's sampled directly from an 8bit ADC, so I suppose rounded would be the most accurate term, although I'm not 100% sure of the details? $\endgroup$
    – Aaron Shifman
    Commented Apr 8, 2016 at 1:39
  • $\begingroup$ Then ... what would smoothing the values tell you about? $\endgroup$
    – Glen_b
    Commented Apr 9, 2016 at 2:46
  • $\begingroup$ @Glen_b What we're ultimately trying to do is find 0-crossings (on scaled data), and the flip-flop behavior makes that quite difficult, because you don't want to take the first or the last crossing necessarily. $\endgroup$
    – Aaron Shifman
    Commented Apr 10, 2016 at 3:03
  • $\begingroup$ I'm still not sure there's enough here to give sensible advice; if you can't even tell if the underlying values are rounded or truncated, what does it actually mean when the recorded value crosses 0? The actual thing it's measuring may not have done so even once. $\endgroup$
    – Glen_b
    Commented Apr 10, 2016 at 6:56

3 Answers 3

Reset to default
1
+50
$\begingroup$

The shape of this signal makes it a candidate for Haar Wavelet Smoothing, where you decompose the signal int a series of Haar waveforms. It looks like you can effectively de-noise this signal by removing the high-frequency Haar components and keeping the low ones.

Improve this answer
$\endgroup$
1
  • $\begingroup$ That looks really interesting/never heard of it before. Thanks, I'll look into it. $\endgroup$
    – Aaron Shifman
    Commented Apr 14, 2016 at 0:30
0
$\begingroup$

I think the simplest method would be moving average, by taking the mean of a group comprising $x$ points before and after each point:

https://en.wikipedia.org/wiki/Moving_average

You could also try more complex techniques like gaussian smoothing, which applies a gaussian distribution rather than a uniform one as above:

https://en.wikipedia.org/wiki/Gaussian_blur

Note: This is not the same as a gaussian convolution filter, which does give a phase shift:

https://en.wikipedia.org/wiki/Gaussian_filter

From the link: "Since the Fourier transform of a Gaussian is another Gaussian, applying a Gaussian blur has the effect of reducing the image's high-frequency components; a Gaussian blur is thus a low pass filter".

Incidentally, you shouldn't get any phase shift as long as your smoothing distribution (whether uniform or gaussian) is symmetric around each point.

Improve this answer
$\endgroup$
0
$\begingroup$

I suggest non-linear filters, the median or the weighted median to start with. It can remove impulse noises, and respect transitions more than linear filters. A fast C/C++ code is available.

If unsure about the size of the filter, it can be iterated, made adaptive, or even multiscale in a non-linear wavelet fashion.

[EDIT] I just discovered PWCTools - The piecewise constant toolbox:

Implementations of algorithms for noise removal from 1D piecewise constant signals, such as total variation and robust total variation denoising, bilateral filtering, K-means, mean shift and soft versions of the same, jump penalization, and iterated medians. It uses a range of solvers including interior-point optimization, adaptive step-size Euler integration and greedy knot placement. If you use this code, please cite: M.A. Little, N.S. Jones (2011), Generalized methods and solvers for noise removal from piecewise constant signals: Parts I and II, Proceedings of the Royal Society A

which implements a quantity of methods. Total variation denoising could be worth looking at, and thee recent A Direct Algorithm for 1D Total Variation Denoising implements a C code, illustrated here:enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.