I have a time series of 70000 data points. I want to separate samples of this time series which have very large values as compared to the time series. How can i threshold sample to separate.

What i am thinking for threshold: if SD of sample is s1 and mean and SD of time series are M and S then if s1>M+S then the sample has larger values is this correct? is this logical? any suggestions?

  • 1
    $\begingroup$ What you are asking here is how to detect outliers, you can find multiple answers for such question on this site: stats.stackexchange.com/questions/121071/… or stats.stackexchange.com/questions/129274/… or stats.stackexchange.com/questions/37865/… , so I would recommend starting with them. $\endgroup$ – Tim May 11 '15 at 11:24
  • $\begingroup$ outliers are the specific values of the sample which crosses certain threshold. but in my case i want to consider the whole sample from my population. is my threshold criteria that s1>m+S is logical? $\endgroup$ – user76816 May 11 '15 at 11:33
  • $\begingroup$ You can take literally any subset of your sample, it is a matter of your decision, there is nothing logical or illogical in it... However, check the answers in the links provided. You'll learn for example that mean is very sensitive to outliers, so more robust statistic could be more appropriate in here. $\endgroup$ – Tim May 11 '15 at 11:37
  • $\begingroup$ You seem to be thinking of a threshold when (SD of sample) > (mean + SD of series). Do you really mean that? If so, it would at best select samples, not individual values. I suspect that you really are asking about value > mean + SD, which is at best an arbitrary threshold which will usually select a large number of values. $\endgroup$ – Nick Cox May 11 '15 at 11:41
  • $\begingroup$ yes i am asking about a threshold when (SD of sample) > (mean + SD of series). i am not sure but does this means that this sample has largely spreader values as compared to the mostly data in population? $\endgroup$ – user76816 May 11 '15 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.