2
$\begingroup$

I'm looking for the ideal independence test of two varaibles of unknown distribution, i.e. a non-parameteric test. Would choose between alternatives based on statistical power.

Few options that came to my mind are Kendall Tau and a test based on Spearman's rho. Chi-squared is an option, although power is a function of an additional parameter (binning), which makes it less compelling.

I wonder what other options are out there and how different options compare in terms of statistical power.

To give more color, I work in trading/finance, where forecasting relationships tend to be just as weak that it is difficult to find them. Once they are stronger, they quickly weaken to adhere to efficient market and no free lunch.

Relationships between variable do not have to be monotonic, but continuity of conditional mean or finite number of non-nontinuities is reasonable to assume.

We can assume iid-ness and typical sample size is 250 - 2500.

Improve this question
$\endgroup$
9
  • $\begingroup$ What's "strong independence"? (or if you meant a strong test of independence, what's a "strong test"?). What do you mean by "strength"? Are you talking about power? Against what alternatives? $\endgroup$
    – Glen_b
    Oct 3, 2015 at 17:35
  • $\begingroup$ @Glen_b, yes, I mean power, thanks for noting. I hope its easier to interpret now, I'm not sure about your last question though. $\endgroup$
    – Mark Horvath
    Oct 3, 2015 at 17:45
  • $\begingroup$ There's an infinite number of ways for things to be dependent. Measures like Spearman and Kendall correlations measure particular (and quite similar) types of dependence (and so have good power in tests when the form of dependence is like the kind of thing they measure) -- but there's all manner of dependence that neither of them pick up. So the question is ... what kinds of dependence do you want power to identify? The kind(s) that those two things can pick up? Or something else, perhaps something more general or something more specific? $\endgroup$
    – Glen_b
    Oct 3, 2015 at 18:19
  • $\begingroup$ I completely agree, we can't have a test which finds all kind of dependencies, I'm looking for ideas though which work often well in practice. Like it happens very rearly that a test of pearson correlation would result in a stronger p-value than Spearman, hence I prefer Spearman (unless I really need to test only linear correlation, which is not the case now). Because the distributions are unknown, Spearman and Kendall work fine as they only look at rankings. I wonder if there is anything more interesting here... $\endgroup$
    – Mark Horvath
    Oct 3, 2015 at 19:25
  • $\begingroup$ You could have a sequence of tests which as n grows would will have power against any form of dependence, but the problem is it will tend to have relatively lower power than one suited to the kind of dependence you want power against. When you say "I wonder if there is anything more interesting here", you still have not identified what kinds of performance would be interesting to you. No test will "work well in practice" against dependence it doesn't have power against, and so again, what forms of dependence do you want power against? ... ctd $\endgroup$
    – Glen_b
    Oct 4, 2015 at 0:52

2 Answers 2

Reset to default
2
$\begingroup$

You may be interested in Hoeffding's independence test, which can be calculated using the R function hoeffd in the Hmisc package, and uses a test statistic resembling that of the Crámer-von Mises goodness of fit. The test is consistent, provided we restrict the alternative hypothesis to the case where the two variables are dependent with a continuous joint distribution function; in other words, it is "powerful" against all such alternatives, given a large enough sample.

Another possibility is to construct a contingency table by splitting the range into bins and then apply the $\chi^2$ test for independence.

Improve this answer
$\endgroup$
1
1
$\begingroup$

The test described in Heller, Heller and Gorfine (2012) detects any form of dependence and is powerful in many dependence situations. The R package to use it is 'HHG' and function 'hhg.test'. Documentation here: https://cran.r-project.org/web/packages/HHG/HHG.pdf

Another popular method is the distance covariance by Székely and Rizzo (2009), with the R package 'energy' and function 'dcov.test'. Documentation found here: https://cran.r-project.org/web/packages/energy/energy.pdf

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.