Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

358 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
141 views

When using GAMs, how much concurvity is too much?

Is there a rule of thumb for excessive concurvity? For example, I've heard that a VIF of $>10$ suggests that multicollinearity may be a problem (e.g., in this CV answer), but I cannot find a ...
1k views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
226 views

Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
62 views

multicollinearity and predictive power

In wikipedia it says ...
203 views

What to do when predictors are proportions that sum up to one?

I have a situation as the one described in the links below: Interpreting proportions that sum to one as independent variables in linear regression Predictor variables sum up to 1 but not necessarily ...
406 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
2k views

What is the definition of the generalized variance inflation factors (GVIF)?

I am currently working on a statistical project at my school, in short, it is about finding the "best" linear regression model to explain the price of houses in a giving community. The model we have ...
901 views

check for multicollinearity: ordinal logistic regression

I'm currently working on a statistics paper for practice and cannot figure out how to control for multicollinearity if almost all my variables are ordinal. VIF doesn't seem to work for instance. It ...
108 views

Cox interactions and co-linearity

I am interested in developing a model to predict survival based on a few predictors: age, sex, and two lab values, albumin and globulin. I have approximately 14,000 deaths in the data. I initially ...
3k views

Multicollinearity: How to convert GVIF^(1/(2df)) values to VIF

I am using the GVIF^(1/(2df)) method in my analyses to check for multicollinearity of my (mainly) categorical variables. However, I am struggling with the cut-off values. For the 'regular' VIF several ...
253 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
715 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
8k views

How to diagnose multicollinearity using the output of vif function in R?

I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model. When I run vif() on my final model, I get <...
6k views

Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
69 views

Can you control for confounding variables by building a seperate primary and residual model?

TLDR: I want to seperate out any possible influence from a set of confounding variables on the response before estimating the effects of the variables of interest. Can I use two sequential models? I ...
214 views

“Regression design matrix is rank deficient” - multicollinearity between two categorical variables?

I have the following model: Reduction_in_clinical_score ~ Baseline_clinical_score + Site_of_data_collection + Treatment_Type + Age + Sex + ERP Site of data ...
43 views

Multicollinearity and Partial Dependence Questions

Assume I build a binary classification model to predict p(y=1) from {x1, x2, ... x10} For now, assume that model could be a GBM, RandomForest, or Logistic Regression. Also assume that all of the ...
40 views

In linear regression, what is the difference between performing variable selection before assessing multicollinearity or vice versa?

If you have a number of variables you're interested in and want to perform linear regression, is there a clear preference between: Method A. Perform variable selection techniques (e.g. using ...
55 views

Can condition number and VIF be used on a model with categorical variables?

I am trying to test whether a model I am using has multicollinearity. The only two methods I've learned are condition numbers and variance inflation factor (VIF) to determine whether multicollinearity ...
191 views

Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
1k views

Difference between suppressor variable and multicollinearity

I have having trouble understanding the difference between a suppressor variable and multicollinearity in multiple regression. If a suppressor variable is one that is not correlated to the outcome, ...
430 views

Identifying variables contributing to near multicollinearty in linear regression using VIF's and multiple R squared's

When trying to detect collinear columns in $X$ a high proportion of cases give a $R_k^2$ close to 1 for independent columns (see figure). When near multicollinearity arise in a $n\times m$ data ...
2k views

Does collinearity largely impact the performance of neural network?

I am build a neural network (NN). Its input include the parameters of a distribution, e.g., mean, median, standard deviation. For example, I am using NN to predict the height of a person. The input ...
76 views

Multiple errors-in-variables regression with collinearities

I have a $[k \times N]$ matrix of predictors / independent variables and a $[k \times N]$ matrix of predictands / dependent variables. I have uncertainty estimates for each predictor and each ...
340 views

Detecting collinearity in Logistic Regression model

I'm running a predictive model using the logistic model in SAS and, currently, I'm trying to perform some diagnostics about the collinearity issue in the estimated model. To do that, I followed step-...
1k views

Shapley Value Regression for prediction

I've been successful in using the relaimpo-Package for R in SPSS through STATS_RELIMP to calculate the Importances of different predictors (in cases of multicollinearity). What im wondering now is how ...
164 views

reconciling Linearity and Multicollinearity assumptions in ANCOVA

For ANCOVA, many textbooks and other resources require, among other assumptions, that covariates be linearly related to the dependent variable, which makes sense. However, many of the same sources ...
354 views

Interpretation of elastic net coefficients under multicollinearity

I am studying the elastic net regression and in some material I read, it was mentioned that the method will choose a group of regressors that are correlated while LASSO can pick one among the ...
380 views

Variable correlation and collinearity in logistic regression

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to ...
203 views

Variance Inflation Factor and Condition Indeces

My data is cross-sectional macroeconomics data. I have six independent variables (x1,x2,x3,x4,x5,x6) plus 2 dummies (d1,d2) plus 2 interactions terms (d1*x1,d2*x1).I am testing my data for ...
903 views

Very low VIF values, but extreme high condition index

In my multiple linear regression model, all of my explanatory variables have a VIF score, lower then 3, but the highest condition index is 709. The constant and one of the explanatory variables have 1....
13k views

VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
194 views

Question about variance inflation factors

I'm considering the regression model $y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \varepsilon_i$ where the $\varepsilon_i$ are iid and $\mathcal N(0,\sigma^2)$ A study question asks to show the ...
142 views

34 views

Can I use compositional data in a GLM?

I have landing data (kg) of five species where I try to identify which factors may be contributing most strongly to the high catch. Before, I had 4 predictor variables (Depth, Chlorophyll, Temperature ...
39 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
194 views

About spurious & masked relationship and multicollinearity

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
13 views

About multicolinearity in Interaction Terms

Dears, In my panel data analysis, I am using interaction term. The model looks something like Y=b0+b1*X+b2*Z+ b3*XZ+e. Now the interaction term is strongly correlated with X or Z. In order to get ...
165 views

Multicollinearity in structural equation modeling with multiple imputation?

Using R, I created a structural equation model and fit it to multiple datasets using the 'sem.mi()' function from the SemTools package. I know multicollinearity tends to be a concern for structural ...
925 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertuned ML trees etc. ...
28 views

High correlation between independent variables in multivariate regression: collinearity?

I will be grateful for some advice regarding using two highly correlated independent variables in a multivariate regression. I would like to control for respondents' religion (Muslim, Christian, ...
211 views

R code for robust ridge regression

I am having trouble in searching for the MSE value in using robust ridge regression. The robust estimators that i used is LTS and MM. However, when both robust estimators were applied to ridge, the ...
222 views

Concurvity in additive modelling. Parametric term?

I am modelling a outcome in gam were two variables (x1 and x2) are continuous and the other four are factors. I have a suspicion that x1 and x2 might be collinear so I want to check that. As I ...
121 views

Is multicolinearity testing altogether useless if the p-value on each regressor is less than 0.01?

Isn't the effect of multicolinearity to artificially increase the standard errors and thus artificially decrease the t-statistics, and thus artificially increase the p-values? If so, if all ...
78 views

VECM with Multicollinearity

I have fit a vector error correction model (VECM) to some macroeconomic data. In particular, I am interested in three relationships real GDP as a function of employment and real wages employment as a ...
56 views

What is the difference between multicollinearity and correlation?

I have a set of data and I wish to construct a multivariate regression model for predicting. I saw that if the variables are multi-colinear the multivariate regression model will be bad. I don't know ...
31 views

How to Interpret Co-Linear Interaction Term?

I am looking to model transit ridership based on demographics at the stop level. For each stop, I know: ONS: Total boardings per day - averaged over the number of times a bus pulls up at the stop ...