# Reducing the number of groups of data by joining them

I have one variable that attains a certain value (between 0 and 1), about 1000 observations and each belongs to one (and only one) of fifteen groups. Now I would like to reduce the number of groups to two or three by combining them. It is important that all the data within a given group in the former division will stay together after the pooling.

The problem is, that the whole dataset has too large variance, so I would prefer several groups that are more focused, less varied.

Example: Say you have hundreds of data on incomes of people from different EU countries. From this, you estimate average salary for each country (just an example). You would like to now create a group of "rich countries" and "poor countries" — but where do you draw the division line? Or two division lines in case of three groups (adding, say, "average countries").

Since yet again I don't know the proper statistical approach, I at least give you what my idea was:

For the reduction to two, instead of trying all possible divisions, I had the idea of sorting the groups by their sample means (from $g_1$ to $g_{15}$ where means satisfy $m_1 \leq m_2 \leq \dots \leq m_{15}$) and creating pool combinations — first combination would be {[$g_1$]; [$g_2, ..., g_{15}$]}, second would be {[$g_1, g_2$]; [$g_3, ..., g_{15}]$} and so on -- this notation means that in the first case the first pool are only values from $g_1$ and the second pool are all other values. The second case the first pool are values from $g_1$ and $g_2$, the rest is in the other pool.

I would then compute the p-value for sample mean equivalence test of these pool combinations. The lowest p-value of these 14 combination tests would mark the "most different" division into pools.

The same could be applied to the three groups, though there would be a higher number of pool combinations and the "diversity test" would have to be constructed quite differently.

Feel free to ignore my scribbling... since I do not know statistics very well, I at least tried an approach that seemed most logical.

Thanks again!

• So you have 1000 observations, each of which is now categorised into into one of 15 groups, which you want to shrink to 2-3. What is your objective in shrinking to a predetermined number of groups (what will you do with the results/conclude from the grouping), and how many variables have you measured that you can group on? – Michelle Feb 1 '12 at 5:03
• It is primarily for simplicity reasons, I have trouble orienting in 15 groups, would rather have less of them so that I can observe other variables (this variable is only one dimension of a dataset). The conclusion would be like in the example -- different pools in terms of "performance" -- in this case the size of the variable in question. – Ondrej Feb 1 '12 at 7:54
• It would help to know what your overall intent is. If you wish to do a multivariate analysis later, this could be time wasted. – Michelle Feb 1 '12 at 7:57
• OK, so there are basically two intentions: First is pooling of high and low performance countries/regions just out of curiosity. The second is that I will perform a regression later and would like to know if these "high performing" countries are statistically different in some other aspect than the low performing. I know I could just use the this variable in a regression instead of this pooling, but I feel like this approach is more "visible" and comprehensible in the output. I guess there will be a problem accuracy as this way some information will be lost. Hope that makes sense. – Ondrej Feb 1 '12 at 10:06
• To sum up, instead of running $y = x_1 + x_2 + performance$ ($performance$ being the exact value of the observed variable), I would like to run $y = x_1 + x_2 + highperforming$, where $highperforming$ would be a dummy. I guess I should be working on something more meaningful :-) – Ondrej Feb 1 '12 at 10:06