I have a question about using experimental design methods for cross validation in a non-conventional approach.
I will use an illustration to explain the question. Imagine you have 100 time series stacked on top of each other and segmented into adjacent interval partitions (we can use 1990, 1991, 1992 for example). Then suppose we take the time series from the first year and transform them into some factor with levels. So, let's say A, B, C represent top, middle, and bottom 33% of performers, respectively. Then, we look at a continuous response variable such as percent returns over the next time interval. Ignoring any limitations for the moment, like unequal variance, we could use ANOVA to tell us whether or not a set of response variables have any statistically different means. Further, imagine we find the response variable corresponding to B is different than A and C.
Now imagine we slide the box forward, and we apply the same method to the next time interval. I would like to have a test that looks at whether the properties of the response variables have the same statistical differences as the prior interval ... And possibly whether the underlying time series (block?) properties have changed. Imagine a two factor interaction plot with two overlapping V formations representing zero statistical difference. What is the appropriate approach to answer this, is it a variation of ANOVA (e.g. 2-Way)? More importantly, is there a specialized area dedicated to looking at the mixed signal time series cross validation perspective from more of an experimental design perspective? Somewhere along the lines of longitudinal analysis combined with cross-sectional analysis, perhaps?
