I am trying to complement a Run-Sequence Plot by some quantitative metric to validate that a dataset has a fixed location. Since the Run-Sequence Plot will be used in the early phases of Exploratory Data Analysis, the mean is used for location. What I am looking for is a unit-free coefficient and a reasonable threshold to decide whether the location (mean) is fixed.
My initial intuition is to split the dataset in samples of equal sizes, compute the means of every samples, then calculate the relative standard deviation for the computed means. From there, the threshold would be set to 0.01, 0.05, or 0.1. Below the threshold, the dataset would be considered as having a fixed location.
Because this quantitative metric would come in direct support of the graphical analysis of the Run-Sequence Plot from which we expect the analyst to visually decide whether the location is fixed, I am assuming that sampling should be done sequentially instead of randomly. If
n is the number of values in a given sample, the first sample would be made of the first
n values used to generate the Run-Sequence Plot, the second sample of the next
n values, and so on.
In order to reduce the risk of having too few samples and the risk of having samples made of too few values, the number of samples would be defined as the floor of the square root of the number of values in the dataset. Therefore, the number of values in a sample would be roughly equal to the number of samples.
My questions are the following:
- Is this run location shifting coefficient valid?
- Is there a better coefficient?
- Is the suggested sampling method acceptable?
- Is the suggested sample size optimum?
Note: a similar method could be used to detect shifts in variation.