I have a data set consisting of a number of time-series segments, i.e each segment contains n number of contiguous samples from a time-series. Are there any good statistical tests to determine how likely two segments come from the same probability distribution?
Nick's words about the disregard for assumptions regarding independence are priceless. His comment that "No answer or comment to date appears to address something utterly elementary" involving the issue of interdependence of observations motivated me to respond. I have implemented Chow's Test for constancy of parameters How to do Chow test for constancy of parameters across 2 groups , originally applied to known groupings to times series data where the break-point is to be found. The software estimates parameters both globally and locally in order to construct the F test. The software treats anomalous data so that they don't distort the local parameters.
In terms of the OP’s question, one might form/identify an ARIMA model that characterizes each of the two segments and local estimation will then provide two sets of parameters and of course two error variances.A global estimation yields the other piece of the puzzle, leading to the Chow Test
No answer or comment to date appears to address something utterly elementary, meaning here fundamental. Values within segments of time series are in general dependent on each other. This is naturally precisely what is examined by (e.g.) autocorrelation or variogram computations, but it does mean that standard tests from mainstream statistics cannot be carried over to comparing segments of time series, except in the not very interesting case of independent observations.
That is, you can do the computations but the P-values and more generally inferential results will not be valid as independence assumptions will usually be inaccurate. This is not absolutely fatal, as with lesser or greater effort reference P-values can be determined by simulation from realistic processes and so forth, but here "lesser or greater" in practice means "greater".
It is true, as clearly explained by @cjohnson318, that standard two-sample tests result in information conveyed by sequence being lost, but that is a different problem.
Note that a test being non-parametric does not address the issue at all, as non-parametric does not mean assumption-free and, broadly speaking, exactly the same assumptions about independence come into play as well as with e.g. t tests.
Some of the blame here lies with teachers and textbook writers of introductory statistics, who -- faced with an already overcrowded curriculum -- usually fail to spell out the independence assumptions behind mainstream tests. Hence while (for example) there is exaggerated awareness of the importance of normality assumptions among users of statistics, there is very widespread lack of awareness of the importance of independence assumptions.
Two books that do an excellent job of explaining the issue are
Box, G.E.P., Hunter, J.S. and Hunter, W.G. 1978 or any later edition. Statistics for experimenters. New York (later Hoboken, NJ): John Wiley.
Miller, R.G. 1986, reissued 1997. Beyond ANOVA. New York: John Wiley (later London: Chapman and Hall).
This is mostly negative, and more positive suggestions will be expected. Broadly speaking, I have to suggest that analysing change within time series requires time series modelling, and there is no escape from that.
I have been struggling with the same sort of task: a time series split into roughly equal segments and I want to know if each of these segments is similar. Plotting them all it is clear that a proportion are artifacts and very different from the rest (and should therefore be excluded), but getting a test to quantify and automate this process is a lot more difficult that it should be.
Cross correlation or cross-covariance (xcorr and xcov in matlab) have the added benefit of testing a range of lags in case the segments are shifted relative to each other.
Chi-squared goodness of fit test - You can use this to test if each segment is from a normal distribution, or from any arbitrary distribution that you care to define. In my case I took the mean of the segments as and tested each segment against this. Clearly, the more artifact segments the less useful this becomes as they will tend to influence the mean more and more.
Just to recap what is in the comments: you have a dataset in which each time segment has its own distribution. Your goal is test which of these time segments come from the same distribution. To start with, you should try to find the peaks in your data and then for each fit a distribution. You can then test each segment using something like the Hellinger or KL against other distributions in the dataset.
Is it reasonable to assume that each time segment has a unique mode, or are there fewer modes than time segments?