Skip to main content
edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
made some notes on efforts so far
Source Link

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these vectors tells a different story, though. A few look like this:

scatter plot showing a clear negative correlation

but the vast majority are clearly spurious correlations arising from a few outliers, with plots that look like this:

a cluster of points at the bottom left, with a few scattered outliers along the left edge and top right of the plot

I want to filter down to just the vectors which seem genuinely associated, but it's not realistic to go through 3000 plots. Are there statisticsAre there statistics I can calculate for each pair of vectors that will help me distinguish between the first and second cases above?

I can calculate for each pair of vectors that will help me distinguish between the firstthought about doing something with (co)kurtosis, and second cases above?have been playing around with that a bit, but haven't been able to figure out what to do with it exactly.

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these vectors tells a different story, though. A few look like this:

scatter plot showing a clear negative correlation

but the vast majority are clearly spurious correlations arising from a few outliers, with plots that look like this:

a cluster of points at the bottom left, with a few scattered outliers along the left edge and top right of the plot

I want to filter down to just the vectors which seem genuinely associated, but it's not realistic to go through 3000 plots. Are there statistics I can calculate for each pair of vectors that will help me distinguish between the first and second cases above?

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these vectors tells a different story, though. A few look like this:

scatter plot showing a clear negative correlation

but the vast majority are clearly spurious correlations arising from a few outliers, with plots that look like this:

a cluster of points at the bottom left, with a few scattered outliers along the left edge and top right of the plot

I want to filter down to just the vectors which seem genuinely associated, but it's not realistic to go through 3000 plots. Are there statistics I can calculate for each pair of vectors that will help me distinguish between the first and second cases above?

I thought about doing something with (co)kurtosis, and have been playing around with that a bit, but haven't been able to figure out what to do with it exactly.

Source Link
Loading