1
$\begingroup$

I have some data representing times series about houses costs in specific areas. Some of the values along the times series (30 points = 30 months) are missing or are totally wrong (huge spikes).

What I am doing right now is to calculate the average and the standard deviation to eliminate first zeros and spikes and replace them with a smooth interpolation from the extremes of the holes. It works fine in some cases but not always.

Is there any better way to fix these problems?

UPDATED

An idea may be the "1-D median filtering" (Media filter)

$\endgroup$

2 Answers 2

1
$\begingroup$

As you well mention, my first approach would be using a 1-D median filter. Such filter goes through each sample of the signal, centers a window of length $N$ (typically an odd number) in it, and replaces its value by the median of the window itself. It is particularly good to remove abnormally large spikes in your signal and/or missing values.

Depending on your platform, the implementation will differ. I use MATLAB a lot and it has already its own built-in function medfilt1. You will find in its documentation a very nice example (I tried to include it, but I cannot post so many links in an answer yet).

$\endgroup$
-1
$\begingroup$

Some ideas:

You could try with filters like moving average, savitsky-golay, etc. Or Interquantilerange(IQR) to remove outliers

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.