Sorry if this has been asked before but I've already done quite a bit of work here and I feel like I'm quite close to an answer.
I am interested in testing whether the PHP function array_key_exists depends on the size of the array. If it did, we should expect that the growth of the computation time for the function would depend on the size of the array. If it does not, then the computation time should be constant. In other words, I want to test the strength of a possible linear relationship.
I measured the computation time for varying sizes of arrays. A few things I see, in ascending order of importance:
- There lots of points (almost half of them) where the time actually decreases as the size increases. This basically gives me grounds to accept the null hypothesis immediately, because any linear relationship would entail a monotonically increasing function of the size.
- Very low standard deviation in the time, indicating that the values are tightly clustered.
- The slope of the simple linear regression is quite small.
- Exceedingly low Pearson coefficient and its square has practically vanished.
If my null hypothesis is that the real Pearson coefficient is zero, namely there is no relationship between the array size and computation time, how small does my sample Pearson coefficient or its square need to be to reject it? What assumptions do I need to make (perhaps acceptable probabilities of Type I and Type II) to do so?
Is it better to formulate the null hypothesis from the Pearson coefficient or from the linear regression slope? Are they about the same as far as explanatory power in this circumstance?