I want to estimate how robust is the result of a simple correlation test (Y~bX+c) to changes in the number and identity of the observations selected to perform it. For that, I have repeated such correlation many times, each time randomly selecting a different number of observations. E.G. My original data set has 20 observations and I have repeated the same correlation for 100 series of 10 to 20 observations.
Now, I would like to summarize the correlation results in a way that it shows the sensitivity of the parameters to changes in the number and identity of the observations included in the correlation. Any hint?
I have thought of two plots:
One would be a scatter plot of the obtained p value against the number of observations in each iteration
Another would be a scatter plot of the obtained p values against the mean X value of the observations (thus representing observations identity)
Would this be valid/enough?
Thanks in advance