1
$\begingroup$

I have a set of drivers that are binary and a concept to measure that contains natural numbers between 1-10.

I'm currently using Kruskal's key driver analysis to determine the relative contribution of each of the drivers. It's discussed as being more robust that Pearson's Correlation by taking into consideration the complete set of drivers and their relative contribution.

However, is the Kruskal's approach still valid when the drivers are binary and the concept to measure are natural numbers between 1 and 10? I thought about switching to using the point biserial correlation, however this is identical to Pearson's R.

My question is: Where do I set the threshold between a 'good' driver and a 'not so good' driver? It's dependent upon the size of the data and also the properties of the data. Calculating the significance using t-tests (ignoring the fact the data may not meet the necessary assumptions of the t-test (that's bundled in with the pearsonr scipy algorithm), denotes all of them to be significant, as they usually will be because even weak drivers will have some correlation, and aren't 'random'. Therefore do I set the 'strong' drivers to have a very low p-value - something that seems kind of arbitrary. Or is there a better algorithm that can distinguish between strong and weak drivers?

Or is it that no algorithm can really determine what a strong driver is? Is it dependent upon other factors relating to the context of the data that is being analysed?

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ I don't think we can decide what constitutes strong/moderate/weak, because it depends on the system you are studying in addition to the statistical properties of the data. A weak driver in a highly controlled lab-based physics experiment may be very strong for a study of messy biological dynamics in natural conditions. Also, as I'm sure you recognise, a threshold is a convenience that shouldn't be taken too seriously. Think about the frequently-made point that p=0.049 and p=0.051 are not meaningfully different despite lying on different sides of a commonly-used threshold. $\endgroup$ – mkt - Reinstate Monica Jul 4 '17 at 13:09
  • 1
    $\begingroup$ Your final question contains a kernel of wisdom that you will find elaborated in the literature on causality. You may find this literature a refreshing departure from purely phenomenological approaches such as Kruskal's. In particular, I commend to you the "causal-statistical distinction" stressed by Judea Pearl in the work I cite in this answer. $\endgroup$ – David C. Norris Jul 5 '17 at 15:00
1
+50
$\begingroup$

Instead of a correlation you could use a chi-square or likelihood ratio statistic (testing the hypothesis that the distribution of the concept, stratified according to driver $x_i$, after correcting for drivers $x_j$, $x_k$..., is different or the same).

Then you can calculate this statistic for each driver based on the average over all permutations, and it will express the degree in which the variable explains a variation in the distribution of the outcome variable.

You can use this averaged value to make your ordering. If you want to express the significance (and don't care about elegance) then you can use a Monte Carlo approach to compare each driver against a large collection of random drivers.

This answer might become less general if some numerical example and more specific background is given.

|cite|improve this answer
$\endgroup$
2
$\begingroup$

I don't think we can decide for you what constitutes strong/moderate/weak, because it depends on the system you are studying in addition to the statistical properties of the data.

A weak driver in a highly controlled lab-based physics experiment may be very strong for a study of messy biological dynamics in natural conditions. Also, as I'm sure you recognise, a threshold is a convenience that shouldn't be taken too seriously. Think about the frequently-made point that p=0.049 and p=0.051 are not meaningfully different despite lying on different sides of a commonly-used threshold.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ How come you copied and pasted your comment from 2 years ago? $\endgroup$ – Mark Ramotowski Jul 24 at 14:43
  • 1
    $\begingroup$ @MarkRamotowski Basically, because of this discussion on Meta: stats.meta.stackexchange.com/questions/832/… which argues that we should not be posting things as comments if they are answers. I think my comment was in some sense an answer, so I'm trying to fix an old error. $\endgroup$ – mkt - Reinstate Monica Jul 24 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.