Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

358 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
141 views

When using GAMs, how much concurvity is too much?

Is there a rule of thumb for excessive concurvity? For example, I've heard that a VIF of $>10$ suggests that multicollinearity may be a problem (e.g., in this CV answer), but I cannot find a ...
5
votes
0answers
1k views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
5
votes
0answers
226 views

Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
4
votes
0answers
62 views

multicollinearity and predictive power

In wikipedia it says ...
4
votes
0answers
203 views

What to do when predictors are proportions that sum up to one?

I have a situation as the one described in the links below: Interpreting proportions that sum to one as independent variables in linear regression Predictor variables sum up to 1 but not necessarily ...
4
votes
0answers
406 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
4
votes
0answers
2k views

What is the definition of the generalized variance inflation factors (GVIF)?

I am currently working on a statistical project at my school, in short, it is about finding the "best" linear regression model to explain the price of houses in a giving community. The model we have ...
4
votes
0answers
901 views

check for multicollinearity: ordinal logistic regression

I'm currently working on a statistics paper for practice and cannot figure out how to control for multicollinearity if almost all my variables are ordinal. VIF doesn't seem to work for instance. It ...
4
votes
0answers
108 views

Cox interactions and co-linearity

I am interested in developing a model to predict survival based on a few predictors: age, sex, and two lab values, albumin and globulin. I have approximately 14,000 deaths in the data. I initially ...
4
votes
0answers
3k views

Multicollinearity: How to convert GVIF^(1/(2df)) values to VIF

I am using the GVIF^(1/(2df)) method in my analyses to check for multicollinearity of my (mainly) categorical variables. However, I am struggling with the cut-off values. For the 'regular' VIF several ...
4
votes
0answers
253 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
4
votes
0answers
715 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
4
votes
0answers
8k views

How to diagnose multicollinearity using the output of vif function in R?

I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model. When I run vif() on my final model, I get <...
4
votes
0answers
6k views

Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
3
votes
0answers
69 views

Can you control for confounding variables by building a seperate primary and residual model?

TLDR: I want to seperate out any possible influence from a set of confounding variables on the response before estimating the effects of the variables of interest. Can I use two sequential models? I ...
3
votes
0answers
214 views

“Regression design matrix is rank deficient” - multicollinearity between two categorical variables?

I have the following model: Reduction_in_clinical_score ~ Baseline_clinical_score + Site_of_data_collection + Treatment_Type + Age + Sex + ERP Site of data ...
3
votes
0answers
43 views

Multicollinearity and Partial Dependence Questions

Assume I build a binary classification model to predict p(y=1) from {x1, x2, ... x10} For now, assume that model could be a GBM, RandomForest, or Logistic Regression. Also assume that all of the ...
3
votes
1answer
40 views

In linear regression, what is the difference between performing variable selection before assessing multicollinearity or vice versa?

If you have a number of variables you're interested in and want to perform linear regression, is there a clear preference between: Method A. Perform variable selection techniques (e.g. using ...
3
votes
0answers
55 views

Can condition number and VIF be used on a model with categorical variables?

I am trying to test whether a model I am using has multicollinearity. The only two methods I've learned are condition numbers and variance inflation factor (VIF) to determine whether multicollinearity ...
3
votes
0answers
191 views

Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
3
votes
0answers
1k views

Difference between suppressor variable and multicollinearity

I have having trouble understanding the difference between a suppressor variable and multicollinearity in multiple regression. If a suppressor variable is one that is not correlated to the outcome, ...
3
votes
0answers
430 views

Identifying variables contributing to near multicollinearty in linear regression using VIF's and multiple R squared's

When trying to detect collinear columns in $X$ a high proportion of cases give a $R_k^2$ close to 1 for independent columns (see figure). When near multicollinearity arise in a $n\times m$ data ...
3
votes
0answers
2k views

Does collinearity largely impact the performance of neural network?

I am build a neural network (NN). Its input include the parameters of a distribution, e.g., mean, median, standard deviation. For example, I am using NN to predict the height of a person. The input ...
3
votes
0answers
76 views

Multiple errors-in-variables regression with collinearities

I have a $[k \times N]$ matrix of predictors / independent variables and a $[k \times N]$ matrix of predictands / dependent variables. I have uncertainty estimates for each predictor and each ...
3
votes
0answers
340 views

Detecting collinearity in Logistic Regression model

I'm running a predictive model using the logistic model in SAS and, currently, I'm trying to perform some diagnostics about the collinearity issue in the estimated model. To do that, I followed step-...
3
votes
0answers
1k views

Shapley Value Regression for prediction

I've been successful in using the relaimpo-Package for R in SPSS through STATS_RELIMP to calculate the Importances of different predictors (in cases of multicollinearity). What im wondering now is how ...
3
votes
0answers
164 views

reconciling Linearity and Multicollinearity assumptions in ANCOVA

For ANCOVA, many textbooks and other resources require, among other assumptions, that covariates be linearly related to the dependent variable, which makes sense. However, many of the same sources ...
3
votes
0answers
354 views

Interpretation of elastic net coefficients under multicollinearity

I am studying the elastic net regression and in some material I read, it was mentioned that the method will choose a group of regressors that are correlated while LASSO can pick one among the ...
3
votes
0answers
380 views

Variable correlation and collinearity in logistic regression

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to ...
3
votes
1answer
203 views

Variance Inflation Factor and Condition Indeces

My data is cross-sectional macroeconomics data. I have six independent variables (x1,x2,x3,x4,x5,x6) plus 2 dummies (d1,d2) plus 2 interactions terms (d1*x1,d2*x1).I am testing my data for ...
3
votes
0answers
903 views

Very low VIF values, but extreme high condition index

In my multiple linear regression model, all of my explanatory variables have a VIF score, lower then 3, but the highest condition index is 709. The constant and one of the explanatory variables have 1....
3
votes
0answers
13k views

VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
3
votes
0answers
194 views

Question about variance inflation factors

I'm considering the regression model $y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \varepsilon_i$ where the $\varepsilon_i$ are iid and $\mathcal N(0,\sigma^2)$ A study question asks to show the ...
3
votes
0answers
142 views

Should I de-mean a predictor variable before a dummy interaction

Suppose I have the following time-series linear model where $\beta$ is misspecified: $Y(t+1) = \alpha + \beta X(t) + \sum_{i=1}^{10000}\gamma_i Z_i(T) + \varepsilon$ where all parameters are in $\...
3
votes
0answers
1k views

Multicollinearity and regression intercept

I have two continuous variables $x_1$ and $x_2$, such that their sum is a constant: $x_1+x_2=c$. Clearly, I cannot run the following OLS model due to perfect multicollinearity: Model (1): $y = \alpha ...
2
votes
0answers
34 views

Can I use compositional data in a GLM?

I have landing data (kg) of five species where I try to identify which factors may be contributing most strongly to the high catch. Before, I had 4 predictor variables (Depth, Chlorophyll, Temperature ...
2
votes
0answers
39 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
2
votes
0answers
194 views

About spurious & masked relationship and multicollinearity

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
2
votes
0answers
13 views

About multicolinearity in Interaction Terms

Dears, In my panel data analysis, I am using interaction term. The model looks something like Y=b0+b1*X+b2*Z+ b3*XZ+e. Now the interaction term is strongly correlated with X or Z. In order to get ...
2
votes
0answers
165 views

Multicollinearity in structural equation modeling with multiple imputation?

Using R, I created a structural equation model and fit it to multiple datasets using the 'sem.mi()' function from the SemTools package. I know multicollinearity tends to be a concern for structural ...
2
votes
0answers
925 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertuned ML trees etc. ...
2
votes
0answers
28 views

High correlation between independent variables in multivariate regression: collinearity?

I will be grateful for some advice regarding using two highly correlated independent variables in a multivariate regression. I would like to control for respondents' religion (Muslim, Christian, ...
2
votes
0answers
211 views

R code for robust ridge regression

I am having trouble in searching for the MSE value in using robust ridge regression. The robust estimators that i used is LTS and MM. However, when both robust estimators were applied to ridge, the ...
2
votes
1answer
222 views

Concurvity in additive modelling. Parametric term?

I am modelling a outcome in gam were two variables (x1 and x2) are continuous and the other four are factors. I have a suspicion that x1 and x2 might be collinear so I want to check that. As I ...
2
votes
0answers
121 views

Is multicolinearity testing altogether useless if the p-value on each regressor is less than 0.01?

Isn't the effect of multicolinearity to artificially increase the standard errors and thus artificially decrease the t-statistics, and thus artificially increase the p-values? If so, if all ...
2
votes
0answers
78 views

VECM with Multicollinearity

I have fit a vector error correction model (VECM) to some macroeconomic data. In particular, I am interested in three relationships real GDP as a function of employment and real wages employment as a ...
2
votes
0answers
56 views

What is the difference between multicollinearity and correlation?

I have a set of data and I wish to construct a multivariate regression model for predicting. I saw that if the variables are multi-colinear the multivariate regression model will be bad. I don't know ...
2
votes
0answers
31 views

How to Interpret Co-Linear Interaction Term?

I am looking to model transit ridership based on demographics at the stop level. For each stop, I know: ONS: Total boardings per day - averaged over the number of times a bus pulls up at the stop ...
2
votes
0answers
152 views

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily ...
2
votes
0answers
187 views

Can I ignore multicollinearity?

I have a dataset with 57 independent variables, many of which are highly correlated with each other. I calculated the VIF numbers and plotted them against the standard errors of the estimated ...

1
2 3 4 5
8