3
$\begingroup$

I am working with python using pandas, and seaborn libraries. I have a dataframe, that I am using for some machine learning. My dataframe has a target variable, along with several other features. Suppose the following data dictionary

Name Type Dtype
f1 ordinal int
f2 ordinal int
f3 nominal categorical
f4 nominal categorical
f5 discrete int
f6 discrete int
f7 continuous int
f8 continuous int
target nominal int

Now on this dataset, I wish to see a correlation matrix, in order to check which features are colinear to each other as well as which features strongly associate to the target variable.

Please guide me with the following questions:

  1. Is using a correlation matrix (with either of the methods such as Spearsman, Kendall, or Pearson) a right approach on this dataset? These correlation methods come from pandas.DataFrame.corr()
  2. Does the choice of using either of the above mentioned methods depend on the features OR on the target variable?
  3. Do we also consider our target variable before making decision about correlation method? Suppose if my target was a continuous one? or even an ordinal one?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Two things: First, correlation is not a good way to check for collinearity. There are collinearity diagnostics available for regression in R and SAS; I'd imagine they are in Python/Pandas as well. Look for condition indexes or variance inflation factors.

Collinearity is a relationship among any set of independent variables. You can have perfect collinearity with low correlation. For an extreme example, imagine that there are 10 IVs. Nine of them are random and unrelated. The tenth is the sum of the other 9.

Second, if you do want correlations for some other reason, then which type of correlation you can do depends on the variable type. With continuous variables you can use any of the methods, although they will do different things. With ordinal you should really only use a ranked method like Spearman's. And for nominal data, correlation doesn't exist (there are other measures of association).

Improve this answer
$\endgroup$
3
  • $\begingroup$ VIF isn't standardized like correlations. In correlation I know lowest and highest values so its kinda more helpful. I know it can only handle colinearity (between pairs). To handle MC, I definitely go for VIF, or do a PCA. Is that correct? $\endgroup$
    – letdatado
    Commented Jan 8 at 13:43
  • $\begingroup$ Coming back to correlations, does it mean that I should plot two correlation matrices. One for my continuous variables and other for my ordinal variables. And for nominal ones, I should do something else. Please advice that as well $\endgroup$
    – letdatado
    Commented Jan 8 at 13:44
  • $\begingroup$ I prefer condition indexes for collinearity. PCA doesn't do it. VIFs are OK. (My dissertation compared VIF and condition indexes and favored the latter). For correlations, you could do one matrix (just with different correlations) or multiple matrices. For nominal-nominal you could use lambda. Also see the association measures tag stats.stackexchange.com/questions/tagged/association-measure $\endgroup$
    – Peter Flom
    Commented Jan 8 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.