# When is it ok to merge data sets [closed]

There are two variables, $x$ and $y$, with $y$ depending on $x$ somewhat. For each occurrence of $x$, I record the value of $y$ so that I get a frequency distribution for $y$ given $x$. (The parentheses are to be read as (value, frequency) for $y$.)

x = 0: (19,2) (20,2) (35,5) (36,7)
x = 1: (14,1) (17,1)
x = 2: (2,1)
x = 3: (18,1) (19,2) (20,1)
x = 4: (27,1) (32,1)
x = 5: (19,1)
x = 7: (19,1)
x = 8: (19,1)
x = 9: (19,1)
x = 10: (19,1)
x = 11: (19,2)
x = 12: (19,3) (20,3) (25,1) (34,1)
x = 13: (19,5) (20,2)
x = 14: (19,3) (20,6) (21,1)
x = 15: (18,1) (19,316) (20,715) (21,4) (27,2) (33,1) (36,1) (37,1) (41,1)
x = 16: (18,2) (19,822) (20,462) (21,3) (22,1) (27,1) (30,1) (31,1) (34,1) (35,1) (39,1) (40,1) (42,1) (43,1)
x = 17: (18,1) (19,2) (20,1) (21,1)
x = 18: (17,1) (20,1)
x = 19: (16,1) (19,1) (20,1) (21,1)
x = 20: (15,1)
x = 23: (13,1)
x = 25: (10,1)
x = 26: (6,1)
x = 28: (6,2) (7,1)
x = 29: (6,1)
x = 32: (3,1)
x = 33: (2,1)
x = 37: (2,1)


As can be seen, $x$ is around 15 or 16 in the vast majority of cases, and the distribution for $y$ is about the same in those cases. So I would like to treat $x = 15$ and $x = 16$ as being the same. Furthermore, it looks as if I can lump more cases together; let's say $x = 12$ to $x = 17$ and conclude that if $12 \le x \le 17$ then we need no further information about $x$ to determine the distribution of $y$ (with some small error). What statistical procedures are relevant here? How can I numerically determine when distributions are similar enough? How can I aggregate the error?

• Since you prefer the answer on math.SE to the ones given here, why don't you delete this question? Jun 27, 2013 at 19:35
• So far, neither version of the question provides us any information useful for determining what constitutes a good answer in the first place. We need an analytical objective, a context, and a description of the data-generation process. Because you maintain the answer on math is "better," in the absence of any of that additional information we must infer you are trying to ask a purely mathematical question, which I agree is appropriate on the math site--but makes it off-topic here unless you clarify or change it.
– whuber
Jun 28, 2013 at 17:50

This is not about merging data sets (at least not in the way I understand those terms to be generally used).

This is about when you can bin values in a distribution within one data set into coarser intervals and the general answer can only be: when that suits your purpose.

I would approach this graphically. Plot your distributions. If the graphs appear to show more detail than you need or can explain, use coarser intervals for your graphs. This kind of control is typical for histogram commands or functions.

But the data should not, in general, be aggregated for any other purpose. To do that is likely to be arbitrary and just to discard information that might be important.

Some older textbooks talk about grouping data, but that's typically in pursuit of short-cuts for calculations that have been superseded by modern software for some decades now.

Otherwise put: If you aggregate your data you will be intellectually obliged to explain that fact, to explain why you did it your way, and to discuss its consequences. Better to avoid all that!

(LATER) I am still fuzzy on what you want but I get these results for # of y, mean y, median y.

      x |         #      mean    median
--------+------------------------------
0 |        16   31.5625        35
1 |         2      15.5      15.5
2 |         1         2         2
3 |         4        19        19
4 |         2      29.5      29.5
5 |         1        19        19
7 |         1        19        19
8 |         1        19        19
9 |         1        19        19
10 |         1        19        19
11 |         2        19        19
12 |         8        22        20
13 |         7  19.28571        19
14 |        10      19.8        20
15 |      1042  19.77639        20
16 |      1299  19.47652        19
17 |         5      19.4        19
18 |         2      18.5      18.5
19 |         4        19      19.5
20 |         1        15        15
23 |         1        13        13
25 |         1        10        10
26 |         1         6         6
28 |         3  6.333333         6
29 |         1         6         6
32 |         1         3         3
33 |         1         2         2
37 |         1         2         2
--------+------------------------------
Total |      2420   19.6314        20

• I don't think this is about coarseness of the intervals. Coarseness here could only relate to the distribution of $y$ given $x = i$, for some $i$, for which I record only discrete integer values and I have no interest in doing otherwise. Coarseness is about combining data points within a distribution in some fashion. Lumping this data with respect to $x$ is combining distributions. Jun 27, 2013 at 18:40
• Then you have lost me, especially given the openness of "in some fashion". You wrote e.g. "I would like to treat x=15 and x=16 as being the same". I call that binning to coarser intervals. Best to re-edit your question if you don't get a better answer. Jun 27, 2013 at 18:43
• I asked this question on math.se as well. The answer I got there is along the lines of what I am looking for. But what you say about being "intellectually obliged to explain that fact" still applies. Jun 27, 2013 at 19:07
• I didn't know about your cross-posting. Had you asked "How can I determine when two histograms are similar?" you would have got a very different answer. Jun 27, 2013 at 19:15
• @thethuthinnang please check the help, which asks you not to cross post Jun 28, 2013 at 6:17