3
$\begingroup$

I am using a moving average filter to smooth data for outlier removal. By changing the number of average points, I am getting different result.

My data are multi-dimensional feature vectors.

I applied the moving average to the entire matrix and then on individual variables.

They give different results.

So, how to choose/guess the number of points to average over and should it be applied on the entire matrix or on a one by one basis?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ One approach to choosing a smoothing parameter would be to optimize one-step-ahead prediction errors (such as sums of squares of one-step-ahead prediction errors). If you're trying to identify outliers, you'd want a different measure of prediction error - one reasonably robust to outliers (and then moving averages would seem an odd choice - why not something more robust to the outliers?) $\endgroup$
    – Glen_b
    Commented Nov 2, 2013 at 1:11

1 Answer 1

Reset to default
3
$\begingroup$

Neither. Both. All.

Sorry. But I think this is another attempt (albeit a clever one) to automate what can't really be automated. Of course different methods give different results; the only times they wouldn't is where the outlier is so obvious that you don't need a test.

My suggestion is to use a variety of methods to identify possible outliers, then examine those outliers on an individual basis.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your reply. But I could not follow what you mean by automating. How to detect presence of outliers and where they are? I just applied outlier removal since existing resources mention that data need to be smoothed and noise-free. My data comes from kinect sensor. $\endgroup$
    – Srishti M
    Commented Nov 1, 2013 at 22:07
  • $\begingroup$ From your question, you seem to be trying to apply a method and use the result, without any further consideration. This would be "automated". Instead, use a variety of methods and then consider the results (the consideration makes it "not automated". $\endgroup$
    – Peter Flom
    Commented Nov 2, 2013 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.