# Hypothesis testing - is my distribution non random?

Say I have $n$ pairs of time series data. For each pair, I calculate a measurement which looks at the overlapping fraction of each time series pair, resulting in $n$ overlap values.

EDIT : I define "overlap" of the time series as follows. For each time series pair, divide one signal by the other (the signals are the same length), resulting in a new signal $d$. Then, given some threshold $x$, look for each instant in time that satisfies $1-x <= d <= 1+x$. Whenever this condition is satisfied, say that the signals are "overlapping". The variable $x$ is a tolerance threshold allowing us to change how close the signals get before we accept them as overlapped (if $d=1$ then the signals are perfectly overlapping at that time instant). The final statistic used tells us the fraction of $d$ that the signals are overlapping - according to the above definition - i.e. it tells us how much (relative) time the signals in each pair are overlapped. It is the distribution of these values that I am interested in.

I would like to determine if the distribution of the overlap values is non-random, i.e. if my time series pairs were somehow randomized, would the distribution of these "random" overlaps be different from the distribution of the "real" overlaps I have already calculated.

Any help would be much appreciated.

• I am not sure what you are asking here. Usually, the term “random” says something about a way a realization of a variable was achieved and is thus not a property that can be tested. If a variable (or, say, a time series) is the realization of a random process, of course also derived properties such as overlaps will be random. – mzuba Jul 5 '13 at 13:32
• @mzuba Each time series pair consists of data obtained experimentally; they are not realizations of a random process. What I am searching for is a relationship between each pair. When I say "random", I mean, is the distribution of overlaps that I actually see the same as one would expect if there were no relationship between the time series? My idea was to randomly select individual time series from each pair and calculate their overlaps (each pair is independent, thus no relationship), then compare this with my actual overlaps from the real pairs. I hope this clarifies the problem. – allhands Jul 5 '13 at 15:31
• Could you precisely define 'overlap' in this context please? – Glen_b Jul 6 '13 at 2:53
• @Glen_b The overlap measure is a straightforward way to measure how close the two time series are. At each time increment, if the signals are within x percent of another (where x is a threshold variable that can be changed), then they are deemed "overlapping". I then take the fraction of time - out of the entire time series - that the signals are overlapped. This gives me the summary value for each time series pair. – allhands Jul 7 '13 at 22:26
• Thanks. I've done a fair bit of time series work and never seen the term used this way. Could you clarify the precise definition of "within $x\%$ of each other" (including, more specifically, what's on the denominator) and edit this definition of the term 'overlap' into your question? – Glen_b Jul 7 '13 at 22:50

Your analysis seems to presuppose that both variables are always positive or always negative. Otherwise either or both of two problems are likely: 1 is not a good reference level for the ratio; or, if either variable could ever be zero, the ratio would be indeterminate some of the time and so useless. A wider problem is that ratios tend to explode whenever denominators are very small.

I can readily believe that this assumption of same sign does not bite for you, but it is a reason why you are not, so far as I can see, moving in a direction that is likely to be at all fruitful. That is, the useful methods tend to be those that can be widely used and are not dependent on some special feature.

This measure appears home-grown. The need to specify a threshold could be a feature if the threshold corresponds to something practically important to you; otherwise it just looks arbitrary, i.e. you have a family of measures and need to think in terms of overlap as a function of allowed threshold.

Note that your measure doesn't actually use the information about time in the data as the overlap would be the same for the same values in different time order. That seems to undermine your idea that you could use the measure to test non-randomness, however that is defined.

An over-arching comment is that you seem to be feeling your way to what is better quantified via correlation, including cross-correlation. The trade-off between reading some literature to find out standard methods and inventing new methods that make intuitive sense can be delicate, but usually the former strategy works better.

• Thanks for your comments. I'll try to respond to each. Our particular time series are always positive (and non zero), and tend to be located fairly close to one another. Without going into too much detail, this is indeed a "home grown" metric we have developed specifically for the purposes of these time series; it is not intended for wider use. With regards to the threshold, it is a feature with practical significance for us, given the means by which the signals are generated. Also, the time information is not significant to us in the sense of ordering the data... – allhands Jul 8 '13 at 14:49
• ...to continue, we have previously explored a number of correlation measures, with varying degrees of success. The problem has been finding a method that can cope with the non stationarity and aperiodocity of our data. For instance, I used this local correlation tracking algorithm, although nothing meaningful could be inferred. The tool that this question concerns is just a simplified way to measure a particular process we are studying. – allhands Jul 8 '13 at 14:55
• The price of wanting to do things differently -- however good the rationale -- is some dissociation from theoretical statistics. I can't suggest how at all you could get significance statements out of this approach. Note that although you are not, it seems, especially interested in the time series aspect here, any significance test for your overlap measure would need to be robust to dependence, nonstationarity etc. – Nick Cox Jul 8 '13 at 15:26
• Your comments are appreciated. Let's say I determined the cross-correlation of each pair instead, and wanted to compare the distribution of these correlations with those of "random" pairs. Would this approach be amenable to significance testing? – allhands Jul 8 '13 at 16:49
• Now this is a new question and any decent book on time series will tell you. – Nick Cox Jul 8 '13 at 18:04

I am not sure if the following is what you want, but should you be interesting in a test for the hypothesis:

“Is my observed realization of random variable X (1 if d > x, 0 otherwise) distributed as defined in expected distribution Y?”, use the Pearson’s Chi Squared test of independence / homogeneity.

Note that this is not a method of time series analysis and will ignore the time dimension of your data, and will furthermore heavily depend on the choice of x.