0
$\begingroup$

I have a sorted list of integers. From this list, I would like to find intervals of numbers in which most of the numbers are concentrated.

I have used K-Means with R and played around with the k parameter to visually identify ranges, but I am not satisfied with the justification of this method as I'm not sure how to evaluate how good each clustering scheme is.

Would it be possible to use something like Kernel Density Estimation to get the ranges?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ Try to be more precise about what is an "interval" for you, and what would be the best solution. One dimensional data is easy, and finding the optimum solution may be in O(n) or O(n^2) if you spell it out, rather than hoping k-means happens to gind qhat you are looking for (if you even know yet, what you really desire). $\endgroup$ – Has QUIT--Anony-Mousse Jan 15 '16 at 9:38
  • 5
    $\begingroup$ See stats.stackexchange.com/questions/163778/… $\endgroup$ – Tim♦ Jan 15 '16 at 14:23
  • $\begingroup$ An interval to me is a pair of minimum and maximum numbers defining a range of numbers. I would like these ranges to cover all of the integers in my sorted list. I would also like the ranges to separate groups of frequently occurring numbers. I thought k-means would make sense here because I would be finding the centers of the intervals I seek. $\endgroup$ – Duck Jan 15 '16 at 14:25
  • $\begingroup$ It's clear whether you're looking for an algorithm or method? $\endgroup$ – Aksakal Jan 15 '16 at 14:30
  • $\begingroup$ Either algorithm or method would be appreciated. $\endgroup$ – Duck Jan 15 '16 at 14:48
0
$\begingroup$

Well, yes, indeed. You could estimate the point density via kernel estimation and what would be likely a smoother density function. Perhaps you could try out the Parzen window methodology.

You'll be facing a trade-off between bias and variance. When assessing the "quality" of your estimate, you'll likely have to decide where you'd like to stand wrt that trade-off (in the case, eg, of the Parzen window, by adjusting the width of the window)

|cite|improve this answer
$\endgroup$
  • $\begingroup$ How would I be able to extract the number intervals after I do a kernel density estimation? $\endgroup$ – Duck Jan 14 '16 at 21:08
  • 1
    $\begingroup$ @Duck You might start with something like a hpd-type interval based off the KDE and then adapt your interval from there, perhaps by looking at where the actual data are near the endpoints $\endgroup$ – Glen_b -Reinstate Monica Jan 15 '16 at 1:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.