3
$\begingroup$

I have some data which I am using to show that there is no relationship between two variables. (Or only a weak one.) In a previous writeup, I included the scatterplot showing no visible relationship, as well as the Pearson's and Spearman's correlation coefficients, which were both low. One of the reviewers commented that the statistics in the paper were weak and we 'only showed some correlations'. Is there any better way to show that there is little/no relationship between these variables? I have transformed the data using every reasonable transformation (log, square root, exponentiation), and even averaged the values for each datapoint, though it seems to me that that is falling prey to the ecological fallacy. Nothing. Not even a hint of a pattern anywhere.

Obviously, I can't show everything I tried. I want to show that there is no (or at least little) relationship, because common wisdom is that there should be a strong one. The fact that there is not a strong relationship is a surprising result. I know that you can't prove the null hypothesis, but I would like to show as much as possible that any other options are unlikely.

How do I convincingly show that there is no relationship between the variables? (Other than hypothesis-testing my correlations, which I am planning to do, but that doesn't show that there is no relationship.)

Note that both variables have heavy-tailed distributions - does that make a difference to the answer?

Below is a scatterplot of my data.

Improve this question
$\endgroup$
3
  • $\begingroup$ It might help if you provided some images of your data or a description of what they actually are. And maybe discuss why you WANT to show that there is no relationship. In general, the burden of proof is on demonstrating a relationship. You can no more PROVE the null hypothesis than you can prove that there is no flying spaghetti monster (aside from laying out a convincing argument related to the lack of evidence, which is what you're really asking about - how to bolster your story). $\endgroup$
    – HFBrowning
    May 23, 2014 at 21:05
  • $\begingroup$ I added some explanation as to why I want to do this - thanks! $\endgroup$
    – bsg
    May 23, 2014 at 21:17
  • $\begingroup$ If "common wisdom" says that there "should" be a relationship, you must address that at the theoretical level with reasoning that is discipline-specific. The stats answers given below help, however the burden of proof relies on discipline-specific theory. There are plenty of spurious correlations between variables, which people acknowledge have no basis in theory so therefore they are just taken to be curious coincidences, not a sign of some un-discovered theory. $\endgroup$
    – rocinante
    May 24, 2014 at 9:09

2 Answers 2

Reset to default
1
$\begingroup$

One approach is to start by determining what you consider to be little relationship (ideally this should be done before any look at the data, possibly having an expert in the field who has not seen your data or summaries help her will make this more objective). For example you might consider any correlations between -0.1 and 0.1 to be "little" or meaningless, or some other boundaries. This region is sometimes called the equivalence region. Now compute a confidence interval on your correlation, if the CI falls completely inside your equivalence or little importance region, then that is strong evidence that if there is a relationship, it is not worth getting excited about or worrying about. If part of the confidence interval extends past your bounds then that implies that there is a chance that the true relationship could be in the interesting region and you need more study/data to narrow the interval.

If you (or the reviewers) are concerned that there may be some transformations of the variables that show more of a relationship, then you could first run the ACE (Alternating Conditional Expectations) algorithm on the data. This tries to find the transformations that give the highest correlation between variables. If ACE cannot find a transformation (or pair of transformations) with a high correlation then there likely are not any reasonably smooth transformations that would.

Edit:

Additionally you may want to consider this paper:

 Buja, A., Cook, D. Hofmann, H., Lawrence, M. Lee, E.-K., Swayne,
 D.F and Wickham, H. (2009) Statistical Inference for exploratory
 data analysis and model diagnostics Phil. Trans. R. Soc. A 2009
 367, 4361-4383 doi: 10.1098/rsta.2009.0120

Including some of the suggested plots where you have permuted to guarantee no real relationship as a comparison may be convincing to the reviewers/audience that your date is consistent with no (or little) relationship.

Improve this answer
$\endgroup$
1
$\begingroup$

If you have enough data, an alternative way would be to estimate the empirical joint distribution, and pit it against the product of the marginal distributions. Here too the null hypothesis would be $f_{XY}(x,y) = f_X(x)f_Y(y)$ (i.e. independence), and you will try to disprove it.

This CV post How to compare joint distribution to product of marginal distributions? elaborates, and the various comments there provide also references you can look up.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.