2
$\begingroup$

I know that I can use a logistic regression model to test if quantitative variables influence the binary variable, but I would like to have an Index, like Pearson correlation index to tell how strong is the dependence between the variables.

$\endgroup$
6
$\begingroup$

There is no free lunch here.

For example, suppose you code your binary variable as 0 and 1. (You could use any two distinct values, but 0 and 1 are the best choices on other grounds.) Now you could calculate the Pearson correlation. It has a meaning; the problem is that the meaning is: How well would a straight line summarize the relationship between these variables? As always, looking at the corresponding scatter plot should help you think what you are doing. And the problem with that is that it's not usually the best way to think about modelling the relationship between a binary variable and a counted or measured variable. The alternative, the so-called linear probability model, has some able and articulate defenders, but that is another story.

By the way, I think binary variables are "quantitative" too. If males are coded 0 and females 1, then averages have meaning too. The average of 0,0,1,1,1 is 0.6 and it's the proportion of females. More generally, we are able to use probabilities directly. That's why 0 and 1 is the best coding choice.

Back to the question: A minor industry in the early decades of the 20th century was coming up with analogues of correlation for cases where one or both variables are categorical. You can find them lurking in shady, dusty corners such as the stranger parts of SPSS. Essentially, however, correlations with categorical variables approached that way is a cul de sac.

Statistics spent much of the late 19th and 20th century growing towards the idea that measuring strength of association is less useful and interesting scientifically and even practically than modelling how variables are related. The revolution isn't over yet (and there are qualifications to this principle). Historically, this is what G.U. Yule saw that his teacher Karl Pearson did not really grasp, for example.

But I think you can have it both ways in more modern style.

Summary of remainder of answer: The root of a generalized $R^2$ can be calculated for generalized linear models. So, you can fit a GLM (in this question, a logit model) but get to see something similar to a correlation that can also be directly related to modelling results.

In particular, Zheng and Agresti (2000) discuss the correlation between the response and the fitted or predicted response as a general measure of predictive power for generalized linear models (GLMs). This measure has the advantages of referring to the original scale of measurement, of being applicable to all types of GLM and of being familiar to many users of statistics. Preferably, it should be used as a comparative measure for different models applied to the same data set, given that restrictions on values of the response may imply limitations on its value (see e.g. Cox and Wermuth, 1992).

For an arbitrary GLM, this correlation is invariant under a location-scale transformation and it is the positive square root of the average proportion of variance explained by the predictors. However, again for an arbitrary GLM, it need not equal the positive square root of other definitions of $R^2$; and it need not be monotone increasing in the complexity of the predictors, although in practice that is common. The correlation is necessarily sensitive to outliers.

As the predicted is a function of the observed, the correlation calculated from a sample may be expected to be biased upwards. A jackknifed correlation is one alternative. Zheng and Agresti provide more discussion of this point, including other estimators and a bootstrap approach to providing confidence intervals for the correlation and to estimating the degree of overfitting.

Cox, D.R. and N. Wermuth. 1992. A comment on the coefficient of determination for binary responses. American Statistician 46: 1-4.

Zheng, B. and A. Agresti. 2000. Summarizing the predictive power of a generalized linear model. Statistics in Medicine 19: 1771-1781.

Note for Stata users: there is an implementation in glmcorr (SSC).

Note for everyone else: it is, or should be, an easy calculation in your favourite software.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.