0
$\begingroup$

Say I have the following sequence:

Figure_1

Is there a way to get a probability for each point indicating whether it is an outlier or not of the underlining strictly non-decreasing sequence?

I suppose the best way would be to find largest strictly non-decreasing subsequence. Once this is found, what statistical tools could I use to evaluate the amount of deviation for each point from the underlying sequence.

sequence = [0, 0, 845, 100, 830, 1358, 100, 166, 170, 176, 200, 200, 224, 228, 240, 280, 346, 350, 357, 382, 436, 454, 524, 524, 544, 550, 560, 560, 570, 588, 594, 606, 632, 642, 684, 660, 733, 746, 755, 555, 800, 800, 868, 876, 891, 900, 905, 911, 924, 932, 891, 590, 956, 1000, 1000, 1012, 1034, 1040, 1046, 742, 1076, 1086, 1128, 1142, 1144, 1176, 1200, 1210, 1232, 1270, 1300, 1326, 1342, 1354, 1368, 1376, 1400, 1450, 1468, 1470, 1482, 1487, 1498, 1499, 1499, 1142, 891, 832, 1475, 1163, 1475, 350, 1400, 1250, 1249, 861, 1250, 1187, 1250]
longest_nondecreasing_subsequence_indices = [0, 1, 3, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 52, 53, 54, 55, 56, 57, 58, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84]
$\endgroup$
2
  • $\begingroup$ Statistics alone will not be able to solve this for you. You will need to define what an outlier is - which will involve making assumptions. $\endgroup$ Feb 9, 2023 at 7:07
  • $\begingroup$ @user2974951 thanks, that's probably more poignant (on the mark) than you might know. I always try to rely on stats magic instead of finding a good way to define outliers. In this case it's easy; the largest sub-sequence. For my pitch detection algorithm; not so easy. $\endgroup$ Feb 9, 2023 at 7:22

1 Answer 1

0
$\begingroup$

You could just interpolate the good subsequence and then compute the variance of the residuals.

I suppose the right thing to do is compute an autoregressive model of the good subsequence, then compute the residuals of the AR model with the original subsequence. But this probably wont be worth it for my immediate purpose.

$\endgroup$
1
  • $\begingroup$ I guess I didn't really need to ask this question. My confidence solving stats these questions is lower than my ability. $\endgroup$ Feb 9, 2023 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.