# Correlation on ordered subset

Imagine a hypothetical scenario in which a ball is thrown along a straight line. During flight, the position is continually sampled; however, at some distance, the sampling fails and only noise is detected. This distance is unknown and variable. One approach to assessing correlation between position samples and time in these data could be to start with a short data segment and increase the number of points considered until a maximum correlation is found. An example is shown in the figure below, with the resulting sequence of correlation values (blue line), the original data (inset, black dots), and the correlation line taken from the maximal correlation coefficient (inset, red line). Note that the iteration here starts only after the first 20 points.

After searching for some time, I am wondering whether this is an accepted technique of which I am not aware. Perhaps there are references in the literature that cover this topic? If not, are there obvious problems or confounds with such an approach? Note further that this question is not a duplicate (e.g., here and here), because I am addressing a specific, ordered subset (but of variable length).

Finally, would it be a correct procedure to take the maximal correlation of such a sequence as a single hypothesis test (i.e., of the type H$_0$: $\rho$ = 0, H$_1$: $\rho$ > 0), or would multiple comparisons be a consideration?

• Linking through the tags I added will provide a lot of relevant information. – whuber Oct 22 '14 at 14:44

I haven't come across anyone using correlation as a measure to detect where the changepoint occurs. I think that for the data you describe, the easiest thing to do is to perform a change in regression. This will easily identify both the changepoint and the slope (or not) of the pre change and post change mean.

In R the strucchange package can easily do this: http://cran.r-project.org/web/packages/strucchange/index.html

The documentation, in my opinion, isn't easy to follow so here is a short example:

set.seed(987234)
y=c(1:50,rep(0,50))+rnorm(100,sd=c(rep(10,50),rep(20,50)))
plot(y,pch=19) # looks similar to your data
library(strucchange)
breakpoints(y~c(1:100))


The last line gives 1 breakpoint with the first observation of the new segment being 51.