0
$\begingroup$

The question might be simple, but I am not able to find the answer. Hence I am asking here. I did search google but didn't get an answer.

I have a continuous stream of data coming from an API in the form of a sine curve (not exactly sine curve) as shown below.

enter image description here

My problem is to generate a single curve which approximates whole data. One problem I am facing is with outlier curves as similar to the middle one.

enter image description here

I am not finding any good leads for my problem. I just need suitable terms so that I can search answer properly for the problem.

$\endgroup$
0
$\begingroup$

Since you have a continuous curve in time, have you considering taking the Fourier Transform of your signal? In this way you can filter it by frequency and eliminate noise such as the lower peak curve, and then transform again to time-domain.

I am in hurry, but I think you will get the idea with this:

You have two curves, as below: enter image description here

In some way you want to differentiate them, taking the fourier transform:

enter image description here

Now in the domain frequency you can see that you can filter some components to take away you yellow signal. Then you can just transform again your signal to time-domain.

Code to generate time domain curve:

import matplotlib.pylab as plt
import numpy as np

%matplotlib inline

x = np.linspace(-np.pi, np.pi, 20)
plt.plot(x, np.sin(x))
plt.plot(x, np.sin(x*np.pi))
plt.xlabel('Angle [rad]')
plt.ylabel('sin(x)')
plt.axis('tight')
plt.show()

Code to the transformed domain:

plt.plot(x, np.abs(np.fft.fft(np.sin(x))))
plt.plot(x, np.abs(np.fft.fft(np.sin(x*np.pi))))
$\endgroup$
  • 1
    $\begingroup$ Nice. I have edited my answer, please let me know if the answer is better now. Also, I think this is the path, you have some things to study right now, let me know if you have any question. $\endgroup$ – Victor Oliveira Mar 6 at 12:46
  • 1
    $\begingroup$ Yes, but that is the point. The noise can be considered as another singla, then when using the frequency domain you can 'split' them. $\endgroup$ – Victor Oliveira Mar 6 at 12:57
  • 2
    $\begingroup$ The sample data clearly do not follow a regular wave-like pattern: the interruption by the small "outlier" curve shifts the phase. This will cause any Fourier-like analysis to fail. $\endgroup$ – whuber Mar 6 at 13:08
  • 2
    $\begingroup$ @whuber yes that's the main problem, I am facing. $\endgroup$ – Rudresha Parameshappa Mar 6 at 13:22
  • 1
    $\begingroup$ DTW may offer a solution. $\endgroup$ – whuber Mar 6 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.