Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

318 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
127 views

When using GAMs, how much concurvity is too much?

Is there a rule of thumb for excessive concurvity? For example, I've heard that a VIF of $>10$ suggests that multicollinearity may be a problem (e.g., in this CV answer), but I cannot find a ...
5
votes
0answers
1k views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
5
votes
0answers
219 views

Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
4
votes
0answers
107 views

Cox interactions and co-linearity

I am interested in developing a model to predict survival based on a few predictors: age, sex, and two lab values, albumin and globulin. I have approximately 14,000 deaths in the data. I initially ...
4
votes
0answers
2k views

Multicollinearity: How to convert GVIF^(1/(2df)) values to VIF

I am using the GVIF^(1/(2df)) method in my analyses to check for multicollinearity of my (mainly) categorical variables. However, I am struggling with the cut-off values. For the 'regular' VIF several ...
4
votes
0answers
383 views

When including a linear interaction between two continuous predictors, should one generally also include quadratic predictors?

Suppose I am fitting a linear model, and I have two continuous predictors x1 and x2. I think that they might interact, so I add ...
4
votes
0answers
237 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
4
votes
0answers
652 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
4
votes
0answers
7k views

How to diagnose multicollinearity using the output of vif function in R?

I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model. When I run vif() on my final model, I get <...
4
votes
0answers
6k views

Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
3
votes
1answer
19 views

In linear regression, what is the difference between performing variable selection before assessing multicollinearity or vice versa?

If you have a number of variables you're interested in and want to perform linear regression, is there a clear preference between: Method A. Perform variable selection techniques (e.g. using ...
3
votes
0answers
30 views

Can condition number and VIF be used on a model with categorical variables?

I am trying to test whether a model I am using has multicollinearity. The only two methods I've learned are condition numbers and variance inflation factor (VIF) to determine whether multicollinearity ...
3
votes
0answers
16 views

Why do my (coefficients, standard errors & CIs, p-values & significance) change when I add a term to my regression model?

Lots of people seem to be asking this. They often seem to get shallow answers that merely assert what is true, instead of drawing or explaining the mechanism. They also seem to not find each other -- ...
3
votes
0answers
1k views

Difference between suppressor variable and multicollinearity

I have having trouble understanding the difference between a suppressor variable and multicollinearity in multiple regression. If a suppressor variable is one that is not correlated to the outcome, ...
3
votes
0answers
384 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
3
votes
0answers
355 views

Identifying variables contributing to near multicollinearty in linear regression using VIF's and multiple R squared's

When trying to detect collinear columns in $X$ a high proportion of cases give a $R_k^2$ close to 1 for independent columns (see figure). When near multicollinearity arise in a $n\times m$ data ...
3
votes
0answers
2k views

Does collinearity largely impact the performance of neural network?

I am build a neural network (NN). Its input include the parameters of a distribution, e.g., mean, median, standard deviation. For example, I am using NN to predict the height of a person. The input ...
3
votes
0answers
68 views

Multiple errors-in-variables regression with collinearities

I have a $[k \times N]$ matrix of predictors / independent variables and a $[k \times N]$ matrix of predictands / dependent variables. I have uncertainty estimates for each predictor and each ...
3
votes
0answers
269 views

Detecting collinearity in Logistic Regression model

I'm running a predictive model using the logistic model in SAS and, currently, I'm trying to perform some diagnostics about the collinearity issue in the estimated model. To do that, I followed step-...
3
votes
0answers
1k views

Shapley Value Regression for prediction

I've been successful in using the relaimpo-Package for R in SPSS through STATS_RELIMP to calculate the Importances of different predictors (in cases of multicollinearity). What im wondering now is how ...
3
votes
0answers
152 views

reconciling Linearity and Multicollinearity assumptions in ANCOVA

For ANCOVA, many textbooks and other resources require, among other assumptions, that covariates be linearly related to the dependent variable, which makes sense. However, many of the same sources ...
3
votes
0answers
344 views

Interpretation of elastic net coefficients under multicollinearity

I am studying the elastic net regression and in some material I read, it was mentioned that the method will choose a group of regressors that are correlated while LASSO can pick one among the ...
3
votes
0answers
373 views

Variable correlation and collinearity in logistic regression

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to ...
3
votes
1answer
178 views

Variance Inflation Factor and Condition Indeces

My data is cross-sectional macroeconomics data. I have six independent variables (x1,x2,x3,x4,x5,x6) plus 2 dummies (d1,d2) plus 2 interactions terms (d1*x1,d2*x1).I am testing my data for ...
3
votes
0answers
12k views

VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
3
votes
0answers
181 views

Question about variance inflation factors

I'm considering the regression model $y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \varepsilon_i$ where the $\varepsilon_i$ are iid and $\mathcal N(0,\sigma^2)$ A study question asks to show the ...
3
votes
0answers
140 views

Should I de-mean a predictor variable before a dummy interaction

Suppose I have the following time-series linear model where $\beta$ is misspecified: $Y(t+1) = \alpha + \beta X(t) + \sum_{i=1}^{10000}\gamma_i Z_i(T) + \varepsilon$ where all parameters are in $\...
3
votes
0answers
1k views

Multicollinearity and regression intercept

I have two continuous variables $x_1$ and $x_2$, such that their sum is a constant: $x_1+x_2=c$. Clearly, I cannot run the following OLS model due to perfect multicollinearity: Model (1): $y = \alpha ...
2
votes
0answers
12 views

About multicolinearity in Interaction Terms

Dears, In my panel data analysis, I am using interaction term. The model looks something like Y=b0+b1*X+b2*Z+ b3*XZ+e. Now the interaction term is strongly correlated with X or Z. In order to get ...
2
votes
0answers
26 views

High correlation between independent variables in multivariate regression: collinearity?

I will be grateful for some advice regarding using two highly correlated independent variables in a multivariate regression. I would like to control for respondents' religion (Muslim, Christian, ...
2
votes
0answers
93 views

What to do when predictors are proportions that sum up to one?

I have a situation as the one described in the links below: Interpreting proportions that sum to one as independent variables in linear regression Predictor variables sum up to 1 but not necessarily ...
2
votes
0answers
330 views

Interpreting interaction term on highly correlated variables

Somebody at a meeting today made the following comment about a Marketing Mix Model (Linear Regression) we run every year. We should account for the high collinearity of the two Marketing Variables (...
2
votes
1answer
132 views

Concurvity in additive modelling. Parametric term?

I am modelling a outcome in gam were two variables (x1 and x2) are continuous and the other four are factors. I have a suspicion that x1 and x2 might be collinear so I want to check that. As I ...
2
votes
0answers
62 views

Is multicolinearity testing altogether useless if the p-value on each regressor is less than 0.01?

Isn't the effect of multicolinearity to artificially increase the standard errors and thus artificially decrease the t-statistics, and thus artificially increase the p-values? If so, if all ...
2
votes
0answers
60 views

VECM with Multicollinearity

I have fit a vector error correction model (VECM) to some macroeconomic data. In particular, I am interested in three relationships real GDP as a function of employment and real wages employment as a ...
2
votes
0answers
136 views

Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
2
votes
0answers
44 views

What is the difference between multicollinearity and correlation?

I have a set of data and I wish to construct a multivariate regression model for predicting. I saw that if the variables are multi-colinear the multivariate regression model will be bad. I don't know ...
2
votes
0answers
30 views

How to Interpret Co-Linear Interaction Term?

I am looking to model transit ridership based on demographics at the stop level. For each stop, I know: ONS: Total boardings per day - averaged over the number of times a bus pulls up at the stop ...
2
votes
0answers
124 views

Why is it necessary to eliminate components in PCR in order to 'solve' multicollinearity?

Running some form of regression on an input dataset that exhibits strong multicollinearity can cause unstable regression coefficients, because the regression algorithm can somewhat arbitrarily ...
2
votes
0answers
57 views

Why financial time series have perfect multicollinearity?

I have daily financial time series of stock returns (35 stocks) which I took the natural logarithm and subtracted the risk-free rate. However, I get the issue non-invertibility of the covariance ...
2
votes
0answers
95 views

Is it appropriate to use a pseudo R-squared when calculating a variance inflation factor for a binary variable?

When calculating the Variance Inflation Factor for a variable $X_j$ in a multiple regression: $$VIF_j= \frac{1}{1-R^2_{j}}$$ if the variable in question is a dummy variable with 0/1 values, is it ...
2
votes
0answers
70 views

Including both Age and Year in mixed effects model?

Study Longitudinal study of forest GrowthRates across time. 28 plots across 80 years, ~14 samples for each plot. Current model: ...
2
votes
0answers
38 views

Regression with correlated variables that cannot be omitted

A short introduction to my question: I have been asked to make a prediction of income/expenditure of a health institution through the year and to incorporate this into a dashboard for the managers of ...
2
votes
0answers
208 views

Multicollinearity between predictors used to simulate data?

I have simulated a dataset containing individual level variables that results from two processes. In the first process there is a selection of individuals according to one variable, say "indQual". ...
2
votes
0answers
126 views

Is it possible to create collinearity issues when creating dummy variables?

I am relatively new to R and stats and am getting a little confused about multicollinearity. I am planning on carrying out ordinal logistic regression, and the majority of my independent variables are ...
2
votes
0answers
2k views

What is the definition of the generalized variance inflation factors (GVIF)?

I am currently working on a statistical project at my school, in short, it is about finding the "best" linear regression model to explain the price of houses in a giving community. The model we have ...
2
votes
0answers
672 views

Variable importance in cases of multicollinearity: OLS vs ridge regression

I have read that when using Ordinary Least Squares (OLS) for multiple linear regression, the coefficients/weights are unreliable for predictor variables that are collinear. I was wondering if this is ...
2
votes
0answers
32 views

How do I know if a predictor changing signs is genuine?

Imagine that predictor A has a positive relationship with the dependent variable and that it also has a high correlation with predictor B. When predictors A and B are entered into a regression model ...
2
votes
0answers
742 views

check for multicollinearity: ordinal logistic regression

I'm currently working on a statistics paper for practice and cannot figure out how to control for multicollinearity if almost all my variables are ordinal. VIF doesn't seem to work for instance. It ...
2
votes
0answers
1k views

Multicollinearity (or not) in exploratory factor analysis

I’m performing an exploratory factor analysis with 28 items, n = 300. I’m confused whether I have a multicollinearity problem or not, and if so whether/how I go about choosing items to remove from the ...

1
2 3 4 5
7