1
$\begingroup$

I want to estimate how robust is the result of a simple correlation test (Y~bX+c) to changes in the number and identity of the observations selected to perform it. For that, I have repeated such correlation many times, each time randomly selecting a different number of observations. E.G. My original data set has 20 observations and I have repeated the same correlation for 100 series of 10 to 20 observations.

Now, I would like to summarize the correlation results in a way that it shows the sensitivity of the parameters to changes in the number and identity of the observations included in the correlation. Any hint?

I have thought of two plots:

One would be a scatter plot of the obtained p value against the number of observations in each iteration

Another would be a scatter plot of the obtained p values against the mean X value of the observations (thus representing observations identity)

Would this be valid/enough?

Thanks in advance

$\endgroup$
4
  • $\begingroup$ why not both? $\endgroup$ Commented May 12, 2015 at 18:16
  • $\begingroup$ I thought of using both, actually. Do you think it would be valid? $\endgroup$ Commented May 12, 2015 at 19:18
  • 2
    $\begingroup$ Graphing data is never _in_valid. You could also plot sample size versus sample mean and label the points 1-20, then plot correlation versus sample label in decreasing order (so if sample 19 has the highest correlation, it is furthest to the left). The real question is: what comparison do you want to emphasize? All graphs are comparisons. Maybe a graph is overkill and a table would suffice. Easy three columns: sample size, sample mean, sample correlation $\endgroup$ Commented May 12, 2015 at 19:22
  • 1
    $\begingroup$ p-values really wouldn't be my choice for a way to summarize or compare regressions; they're random quantities that don't convey much information of interest. If you're interested in impact on correlations, plot corrrelations. $\endgroup$
    – Glen_b
    Commented May 12, 2015 at 22:46

1 Answer 1

1
$\begingroup$

From the comments:

Graphing data is never invalid. You could also plot sample size versus sample mean and label the points 1-20, then plot correlation versus sample label in decreasing order (so if sample 19 has the highest correlation, it is furthest to the left). The real question is: what comparison do you want to emphasize? All graphs are comparisons. Maybe a graph is overkill and a table would suffice. Easy three columns: sample size, sample mean, sample correlation

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.