I would like to measure the accuracy of an algorithm at predicting a certain target value. Since there is some variance in the algorithm's output, I've run it a few times and computed the Pearson correlation between the predictions and the targets each time. Afterwards, I took the average of the correlations using Fisher z-transform.
Can I compute the p-value for this averaged correlation the standard way for Pearson correlations with t-distribution? (i.e. http://www.jeremymiles.co.uk/usingstatistics/chapter8/sigofrexcel.html) The number of samples is the same each time the algorithm is run (the target values are the same), so there does not appear to be any weighting required.
There were quite a few related questions, but none of them seemed to have the particular answer I was looking for: