I'm reading over a report, and have come to a the section on how data was analyzed. There were 26 different features measured at each datapoint, and the author says

If a pair of correlated features were identified, the feature that was most correlated with the remaining features was eliminated, leaving 17 features

Does this mean that if two different features were shown to be correlated to a certain extent, they didn't consider the one most correlated with the rest of the data? How does this help?


1 Answer 1


My guess... what they are talking about is the multicollinearity diagnostic $VIF_k$ (Variance Inflation Factor) of any variable $X_k$, which is defined to be $1/(1-R_k^2)$ where $R_k^2$ is the "R squared" between the kth variable and the other variables. When the variables are explanatory variables in a regression, a high VIF flags explanatory variables that are redundant because they can be described almost perfectly by linear combinations of the other explanatory variables. Then these redundant variables can be tossed.

If you don't toss them, you can get confusion results such as a significant regression relationship (the X variables predict the Y variable) but each explanatory variable $X_k$ appears to have a no significant ($Ho$: slope = 0) effect on the response variable.

If they aren't discussing a VIF, it sounds like they are doing something similar, where they toss variables that have high (multiple) correlation with other variables and thus are somewhat redundant.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.