# Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

358 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0answers
141 views

### When using GAMs, how much concurvity is too much?

Is there a rule of thumb for excessive concurvity? For example, I've heard that a VIF of $>10$ suggests that multicollinearity may be a problem (e.g., in this CV answer), but I cannot find a ...
0answers
1k views

### Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0answers
226 views

### Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
0answers
62 views

### multicollinearity and predictive power

In wikipedia it says ...
0answers
203 views

### What to do when predictors are proportions that sum up to one?

I have a situation as the one described in the links below: Interpreting proportions that sum to one as independent variables in linear regression Predictor variables sum up to 1 but not necessarily ...
0answers
406 views

### What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
0answers
2k views

### What is the definition of the generalized variance inflation factors (GVIF)?

I am currently working on a statistical project at my school, in short, it is about finding the "best" linear regression model to explain the price of houses in a giving community. The model we have ...
0answers
901 views

### check for multicollinearity: ordinal logistic regression

I'm currently working on a statistics paper for practice and cannot figure out how to control for multicollinearity if almost all my variables are ordinal. VIF doesn't seem to work for instance. It ...
0answers
108 views

### Cox interactions and co-linearity

I am interested in developing a model to predict survival based on a few predictors: age, sex, and two lab values, albumin and globulin. I have approximately 14,000 deaths in the data. I initially ...
0answers
3k views

### Multicollinearity: How to convert GVIF^(1/(2df)) values to VIF

I am using the GVIF^(1/(2df)) method in my analyses to check for multicollinearity of my (mainly) categorical variables. However, I am struggling with the cut-off values. For the 'regular' VIF several ...
0answers
253 views

### Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
0answers
715 views

### Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
0answers
8k views

### How to diagnose multicollinearity using the output of vif function in R?

I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model. When I run vif() on my final model, I get <...
0answers
6k views

### Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
0answers
69 views

### Can you control for confounding variables by building a seperate primary and residual model?

TLDR: I want to seperate out any possible influence from a set of confounding variables on the response before estimating the effects of the variables of interest. Can I use two sequential models? I ...
0answers
214 views

### “Regression design matrix is rank deficient” - multicollinearity between two categorical variables?

I have the following model: Reduction_in_clinical_score ~ Baseline_clinical_score + Site_of_data_collection + Treatment_Type + Age + Sex + ERP Site of data ...
0answers
43 views

### Multicollinearity and Partial Dependence Questions

Assume I build a binary classification model to predict p(y=1) from {x1, x2, ... x10} For now, assume that model could be a GBM, RandomForest, or Logistic Regression. Also assume that all of the ...
1answer
40 views

### In linear regression, what is the difference between performing variable selection before assessing multicollinearity or vice versa?

If you have a number of variables you're interested in and want to perform linear regression, is there a clear preference between: Method A. Perform variable selection techniques (e.g. using ...
0answers
55 views

### Can condition number and VIF be used on a model with categorical variables?

I am trying to test whether a model I am using has multicollinearity. The only two methods I've learned are condition numbers and variance inflation factor (VIF) to determine whether multicollinearity ...
0answers
191 views

### Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
0answers
1k views

### Difference between suppressor variable and multicollinearity

I have having trouble understanding the difference between a suppressor variable and multicollinearity in multiple regression. If a suppressor variable is one that is not correlated to the outcome, ...
0answers
430 views

### Identifying variables contributing to near multicollinearty in linear regression using VIF's and multiple R squared's

When trying to detect collinear columns in $X$ a high proportion of cases give a $R_k^2$ close to 1 for independent columns (see figure). When near multicollinearity arise in a $n\times m$ data ...
0answers
2k views

### Does collinearity largely impact the performance of neural network?

I am build a neural network (NN). Its input include the parameters of a distribution, e.g., mean, median, standard deviation. For example, I am using NN to predict the height of a person. The input ...
0answers
76 views

### Multiple errors-in-variables regression with collinearities

I have a $[k \times N]$ matrix of predictors / independent variables and a $[k \times N]$ matrix of predictands / dependent variables. I have uncertainty estimates for each predictor and each ...
0answers
340 views

### Detecting collinearity in Logistic Regression model

I'm running a predictive model using the logistic model in SAS and, currently, I'm trying to perform some diagnostics about the collinearity issue in the estimated model. To do that, I followed step-...
0answers
1k views

### Shapley Value Regression for prediction

I've been successful in using the relaimpo-Package for R in SPSS through STATS_RELIMP to calculate the Importances of different predictors (in cases of multicollinearity). What im wondering now is how ...
0answers
164 views

### reconciling Linearity and Multicollinearity assumptions in ANCOVA

For ANCOVA, many textbooks and other resources require, among other assumptions, that covariates be linearly related to the dependent variable, which makes sense. However, many of the same sources ...
0answers
354 views

### Interpretation of elastic net coefficients under multicollinearity

I am studying the elastic net regression and in some material I read, it was mentioned that the method will choose a group of regressors that are correlated while LASSO can pick one among the ...
0answers
380 views

### Variable correlation and collinearity in logistic regression

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to ...
1answer
203 views

### Variance Inflation Factor and Condition Indeces

My data is cross-sectional macroeconomics data. I have six independent variables (x1,x2,x3,x4,x5,x6) plus 2 dummies (d1,d2) plus 2 interactions terms (d1*x1,d2*x1).I am testing my data for ...
0answers
903 views

### Very low VIF values, but extreme high condition index

In my multiple linear regression model, all of my explanatory variables have a VIF score, lower then 3, but the highest condition index is 709. The constant and one of the explanatory variables have 1....
0answers
13k views

### VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
0answers
194 views

### Question about variance inflation factors

I'm considering the regression model $y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \varepsilon_i$ where the $\varepsilon_i$ are iid and $\mathcal N(0,\sigma^2)$ A study question asks to show the ...
0answers
142 views

0answers
34 views

### Can I use compositional data in a GLM?

I have landing data (kg) of five species where I try to identify which factors may be contributing most strongly to the high catch. Before, I had 4 predictor variables (Depth, Chlorophyll, Temperature ...
0answers
39 views

### Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
0answers
194 views

### About spurious & masked relationship and multicollinearity

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
0answers
13 views

### About multicolinearity in Interaction Terms

Dears, In my panel data analysis, I am using interaction term. The model looks something like Y=b0+b1*X+b2*Z+ b3*XZ+e. Now the interaction term is strongly correlated with X or Z. In order to get ...
0answers
165 views

### Multicollinearity in structural equation modeling with multiple imputation?

Using R, I created a structural equation model and fit it to multiple datasets using the 'sem.mi()' function from the SemTools package. I know multicollinearity tends to be a concern for structural ...
0answers
925 views

### Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertuned ML trees etc. ...
0answers
28 views

### High correlation between independent variables in multivariate regression: collinearity?

I will be grateful for some advice regarding using two highly correlated independent variables in a multivariate regression. I would like to control for respondents' religion (Muslim, Christian, ...
0answers
211 views

### R code for robust ridge regression

I am having trouble in searching for the MSE value in using robust ridge regression. The robust estimators that i used is LTS and MM. However, when both robust estimators were applied to ridge, the ...
1answer
222 views

### Concurvity in additive modelling. Parametric term?

I am modelling a outcome in gam were two variables (x1 and x2) are continuous and the other four are factors. I have a suspicion that x1 and x2 might be collinear so I want to check that. As I ...
0answers
121 views

### Is multicolinearity testing altogether useless if the p-value on each regressor is less than 0.01?

Isn't the effect of multicolinearity to artificially increase the standard errors and thus artificially decrease the t-statistics, and thus artificially increase the p-values? If so, if all ...
0answers
78 views

### VECM with Multicollinearity

I have fit a vector error correction model (VECM) to some macroeconomic data. In particular, I am interested in three relationships real GDP as a function of employment and real wages employment as a ...
0answers
56 views

### What is the difference between multicollinearity and correlation?

I have a set of data and I wish to construct a multivariate regression model for predicting. I saw that if the variables are multi-colinear the multivariate regression model will be bad. I don't know ...
0answers
31 views

### How to Interpret Co-Linear Interaction Term?

I am looking to model transit ridership based on demographics at the stop level. For each stop, I know: ONS: Total boardings per day - averaged over the number of times a bus pulls up at the stop ...
0answers
152 views

### Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily ...
0answers
187 views

### Can I ignore multicollinearity?

I have a dataset with 57 independent variables, many of which are highly correlated with each other. I calculated the VIF numbers and plotted them against the standard errors of the estimated ...