0
$\begingroup$

I would like to somehow use the standard deviation of the following set of numbers to categorize values into five different groups. How would I do this? Do I take the max value of the set and divide by four, to get five equal parts?

Set of numbers 1
1
1
15
22
9
99
100
240
3
7

Standard Deviation
75

Divided by 4 (creates 5 groups)

  • 60 - numbers less than 60 equal category 1
  • 120 - numbers less than 120 and greater and equal to 60 equals category 2
  • 180 - numbers less than 180 and greater and equal to 120 equals category 3
  • 240 - numbers less than 240 and greater and equal to 180 equals category 4
  • numbers greater than 240 equals category 5
Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ There seems to be a bit of confusion here because dividing the range into five equal parts has nothing to do with the standard deviation. You might enjoy researching the clustering questions on this site, with especial attention to "K-means" solutions (which do involve standard deviations). On the Web you could also research "Jenks' method" or "natural breaks classification," which is the one-dimensional version of K-means rediscovered by cartographers to group data into classes for mapping purposes. (The 5-class solution is (1,1,3), (7,9), (15,22), (99,100), (240).) $\endgroup$
    – whuber ♦
    Commented Apr 18, 2012 at 18:39

1 Answer 1

Reset to default
1
$\begingroup$

My idea would be this:

  1. subtract its mean from the set
  2. categorise into bands of multiples of the standard deviation

In the programming language Python this would look something like this:

# load array- and absolute-value function from numerical package
from numpy import array, abs

# have data available in an array
s = array([1,1,15,22,9,99,100,240,3,7])

# calculate moments
mu = s.mean()
sigma = s.std()  # standard deviation

# standardise and categorise
z = s - mu  # subtract mu from each element

s1 = s[abs(z)<0.4*sigma]  # get elements with abs value < 0.4*sigma
s2 = s[(abs(z)>0.4*sigma) & (abs(z)<0.5*sigma)]
s3 = s[(abs(z)>0.5*sigma) & (abs(z)<0.6*sigma)]
s4 = s[(abs(z)>0.6*sigma) & (abs(z)<0.7*sigma)]
s5 = s[abs(z)>0.7*sigma]

This results in:

In [36]: s1
Out[36]: array([22])

In [37]: s2
Out[37]: array([15])

In [38]: s3
Out[38]: array([9, 7])

In [39]: s4
Out[39]: array([  1,   1,  99, 100,   3])

In [40]: s5
Out[40]: array([240])

s1-s5 correspond to the categories. Maybe this is an inspiration for something you like.

Improve this answer
$\endgroup$
4
  • $\begingroup$ thanks for the sample! One question, how did you come up with 0.4 for checking against $\endgroup$
    – chobo
    Commented Apr 18, 2012 at 20:56
  • $\begingroup$ i just fiddled around with the factors to sigma until I had some of the numbers in each category. I started out with 0.2, 0.4, ...; then 0.25, 0.5, ...; and finally 0.4, 0.5, ... $\endgroup$
    – Konsta
    Commented Apr 18, 2012 at 21:43
  • 4
    $\begingroup$ Unfortunately, basing cutpoints on the SD does an awful job when data are highly skewed: it will cram most of the data into one or two bins and leave a lot of empty bins. When you use an arbitrary nonlinear method such as breaking into $[0,0.4]$ sigmas, then $[0.4,0.5]$ sigmas, etc., the result can be deceptive. (When I see data presented in such ways it raises flags suggesting someone is trying to distort the summary or they are just ignorant of effective methods to summarize values. You probably don't fall into either category so you don't want your work to be taken that way, either.) $\endgroup$
    – whuber ♦
    Commented Apr 18, 2012 at 22:08
  • $\begingroup$ Any better alternatives? $\endgroup$
    – chobo
    Commented Apr 19, 2012 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.