Combining many datasets to increase confidence I have a few (5-6) data sets, each is a function of time, with the time span the same between datasets.  These datasets are all statistics of various perspectives of something (the partitions of a graph), and I am trying to find points in time which show interesting changes across the different datasets.  Normally these interesting features are local minima, but this may not always be the case.
So for example, dataset1. is the variance of some measure and at time 0.4 it has a local minima.  dataset2. is the number of maxima and it also has a local minima at 0.4.
The problem is in reality sometimes a minima will show on all datasets and sometimes only on one or a few datasets.  So my goal is basically to combine the information from all datasets to increase my confidence that a local minima at a particular time, is an interesting and significant point.
A simple method would just be to count the number of datasets which have a local minima at that time and and use some threshold value, above which I define the minima to be significant.  But I am wondering if there are more intelligent ways, i.e. ways which can see which datasets are more robust and reliable etc.
 A: I will outline an approach that requires no "training" at all; it is up to you to determine its utility in this case.
A simple (and nonparametric) hypothetical model is that all datasets are independent, that none has a trend, and that their variations from one time period to the next are mutually independent.  This implies the probability with which two pre-specified datasets simultaneously have local minima would equal the product of the probabilities with which each has local minima, with obvious (but more complex) generalizations to three or more pre-specified datasets (which I illustrate below).  In particular, you can estimate the probabilities of local minima by means of their observed frequencies in each dataset.  From these you can compute the probabilities of simultaneous local minima among 2, 3, or, generally, $k$ or more of the datasets.  When the probability for $k$ or more is so low that it is unlikely to occur during the time span you have observed, you can take the simultaneous occurrence of $k$ or more local minima to be "significant" relative to this null hypothesis of independence.
For example, suppose you have five datasets, each observed 100 times, with local minima appearing 8, 9, 10, 11, and 12 times in them.  All five would simultaneously exhibit a local minimum (8/100) * (9/100) * (10/100) * (11/100) * (12/100) = 0.00095% of the time, so even within 100 observations the expected number of simultaneous minima (of 100 * 0.00095% = 0.00095) is so ridiculously low that five simultaneous minima surely would be significant evidence of an "interesting" point.
Local minima among the first four datasets (unaccompanied by a local minimum in the fifth) would have an expected frequency of 100 * (8/100) * (9/100) * (10/100) * (11/100) * ((100-12)/100) = .00697.  Similarly we could compute the expected frequency of local minima among the other combinations of four of the datasets.  The total frequency of exactly four simultaneous minima is 0.04375. Added to the frequency of five simultaneous minima this gives 0.0447 as the expected number of times you would observe four or five simultaneous local minima in 100 observations: still pretty rare and therefore significant if it turns up.  A similar computation for the ten combinations of three simultaneous local minima shows that you would expect at least three local minima 0.8452 times out of 100.  So, observing one or two such events would not be unusual and you might not consider them significant.  Obviously the expected number of two-way minima would be substantial (you should expect to see around 40 of them out of 100) and you would be unlikely to consider any of those significant.
The example illustrates how you could go about computing thresholds for significance in terms of the number of simultaneous local minima for any number of datasets that are observed for any number of time periods.
You can give a more precise accounting of the situation by means of the Poisson distribution.  Take the occurrence of four or more simultaneous minima in the example.  Under the null hypothesis (of independent datasets), this is rare enough that the actual count should have a Poisson distribution with expectation 0.8452.  This implies there is a 94.59% chance of observing two or fewer such events.  Thus, if you see three or more three- or four- or five-way minima you could take this to be significant evidence of lack of independence (with about 95% confidence).  However, in this case you could not point to a specific time that is significant; you could only say that there are more threefold minima than there should be.  Any one of them would be a reasonable candidate for an "interesting" time, but further investigation should ensue before you stipulate that any particular one of these times really demonstrates a departure from independence.
This model might or might not be appropriate for your data.  You can check that by examining the data.  If your data have trends or exhibit serial correlation you would need a more complex version of this model.  Nevertheless, the same kind of analysis can help you decide what constitutes an "interesting" or "significant" syzygy of local minima.
