I have a data set consisting of a number of time-series segments, i.e each segment contains n number of contiguous samples from a time-series. Are there any good statistical tests to determine how likely two segments come from the same probability distribution?
$\begingroup$
$\endgroup$
7
-
$\begingroup$ In order to test for likelihood that 2 segments are both from the same distribution, you will have to first estimate that distribution. If you really need a non-parametric approach that doesn't rely on estimating distributions, I would recommend you look for changes in some representative value, such as the change between means of each time segment. $\endgroup$– learnerFeb 25, 2013 at 15:00
-
$\begingroup$ What about divergence tests like hellinger distance and kullback-leibler divergences? Are there non-parametric versions of these? $\endgroup$– gamerxFeb 25, 2013 at 17:13
-
1$\begingroup$ A fundamental concept to keep in mind is that all probability distributions have parameters (even the uniform distribution). As a result, tests measuring the distance between distributions, like the KL, need distributions to be fitted in order to have something to measure at all. $\endgroup$– learnerFeb 25, 2013 at 17:36
-
$\begingroup$ Also, where is your data coming from? Have you visualized the data collected so far to see if it looks like any particular distribution? Is it bounded in any way? $\endgroup$– learnerFeb 25, 2013 at 17:42
-
1$\begingroup$ Do you mean that each time segment has its own mode? If so, I suppose I might try to identify the peaks first (see this question for some leads) and then try to fit a distribution to each of the identified peaks. $\endgroup$– learnerFeb 26, 2013 at 13:02
|
Show 2 more comments
5 Answers
$\begingroup$
$\endgroup$
2
Nick's words about the disregard for assumptions regarding independence are priceless. His comment that "No answer or comment to date appears to address something utterly elementary" involving the issue of interdependence of observations motivated me to respond. I have implemented Chow's Test for constancy of parameters How to do Chow test for constancy of parameters across 2 groups , originally applied to known groupings to times series data where the break-point is to be found. The software estimates parameters both globally and locally in order to construct the F test. The software treats anomalous data so that they don't distort the local parameters.
In terms of the OP’s question, one might form/identify an ARIMA model that characterizes each of the two segments and local estimation will then provide two sets of parameters and of course two error variances.A global estimation yields the other piece of the puzzle, leading to the Chow Test
answered Jul 31, 2013 at 16:20
-
-
1$\begingroup$ I am very glad to have the support of a time series expert here. $\endgroup$– Nick CoxJul 31, 2013 at 17:57
$\begingroup$
$\endgroup$
No answer or comment to date appears to address something utterly elementary, meaning here fundamental. Values within segments of time series are in general dependent on each other. This is naturally precisely what is examined by (e.g.) autocorrelation or variogram computations, but it does mean that standard tests from mainstream statistics cannot be carried over to comparing segments of time series, except in the not very interesting case of independent observations.
That is, you can do the computations but the P-values and more generally inferential results will not be valid as independence assumptions will usually be inaccurate. This is not absolutely fatal, as with lesser or greater effort reference P-values can be determined by simulation from realistic processes and so forth, but here "lesser or greater" in practice means "greater".
It is true, as clearly explained by @cjohnson318, that standard two-sample tests result in information conveyed by sequence being lost, but that is a different problem.
Note that a test being non-parametric does not address the issue at all, as non-parametric does not mean assumption-free and, broadly speaking, exactly the same assumptions about independence come into play as well as with e.g. t tests.
Some of the blame here lies with teachers and textbook writers of introductory statistics, who -- faced with an already overcrowded curriculum -- usually fail to spell out the independence assumptions behind mainstream tests. Hence while (for example) there is exaggerated awareness of the importance of normality assumptions among users of statistics, there is very widespread lack of awareness of the importance of independence assumptions.
Two books that do an excellent job of explaining the issue are
Box, G.E.P., Hunter, J.S. and Hunter, W.G. 1978 or any later edition. Statistics for experimenters. New York (later Hoboken, NJ): John Wiley.
Miller, R.G. 1986, reissued 1997. Beyond ANOVA. New York: John Wiley (later London: Chapman and Hall).
This is mostly negative, and more positive suggestions will be expected. Broadly speaking, I have to suggest that analysing change within time series requires time series modelling, and there is no escape from that.
answered Jul 31, 2013 at 15:50
$\begingroup$
$\endgroup$
2
I have been struggling with the same sort of task: a time series split into roughly equal segments and I want to know if each of these segments is similar. Plotting them all it is clear that a proportion are artifacts and very different from the rest (and should therefore be excluded), but getting a test to quantify and automate this process is a lot more difficult that it should be.
Suggestions:
Cross correlation or cross-covariance (xcorr and xcov in matlab) have the added benefit of testing a range of lags in case the segments are shifted relative to each other.
Chi-squared goodness of fit test - You can use this to test if each segment is from a normal distribution, or from any arbitrary distribution that you care to define. In my case I took the mean of the segments as and tested each segment against this. Clearly, the more artifact segments the less useful this becomes as they will tend to influence the mean more and more.
-
$\begingroup$ I need a non-parametric or distribution free method though. For my problem, I can't assume that the series follow a normal or any pre-defined distribution. $\endgroup$– gamerxFeb 25, 2013 at 12:14
-
1$\begingroup$ Same here, which is why I ended up using the mean of the segments to test against. In my specific application I can assume that the majority of segments will be valid and only a minority are artifact. Therefore, the mean of the segments will generally reflect a valid waveform. Hence comparing each against the mean will find those that have similar distribution and can be considered valid. $\endgroup$– rgmFeb 25, 2013 at 14:42
$\begingroup$
$\endgroup$
1
Just to recap what is in the comments: you have a dataset in which each time segment has its own distribution. Your goal is test which of these time segments come from the same distribution. To start with, you should try to find the peaks in your data and then for each fit a distribution. You can then test each segment using something like the Hellinger or KL against other distributions in the dataset.
Is it reasonable to assume that each time segment has a unique mode, or are there fewer modes than time segments?
-
$\begingroup$ There could be fewer modes than time segments. $\endgroup$– gamerxFeb 27, 2013 at 5:58
$\begingroup$
$\endgroup$
Have you considered using the two sample rank sums test? That is non-parametric. This will treat your segments as simple data sets, and not time series, so you will lose all information regarding sequence.
