# Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

926 questions
Filter by
Sorted by
Tagged with
5 views

### macroeconomic regressors as xreg in ARIMA - differencing required?

I'm forecasting a timeseries that has both trend and seasonality component, which is why I am using ARIMA. Without providing external regressors, the best model selected (in training) has the ...
31 views

### Why do the coefficients of cross-sectional fixed effects and time fixed effects become zero?

I currently have a panel data set that contains the quarterly increment of loans initiated from more than 300 cities in China over the period from 2011Q1 to 2020Q2. I want to examine the impact of ...
30 views

### How many observations should a dummy variable encompass?

So I feel like this is a rather stupid question, but I can't find a straightforward explanation on the issue. When constructing a dichotomous dummy variable, how many observations should each category ...
74 views

### Is it okay to residualize a variable out of my dependent variable, to deal with multicollinearity?

Here is my situation. I have n predictors of interest, and two control variables. If I put them all together in a multiple regression, I get issues with colinearity (i.e., VIFs are very high, and the ...
22 views

### Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
137 views

### Omitted Variable Bias (OVB) and multicollinearity

In a linear regression model, the reason we control for variables is to prevent the omitted variable bias (OVB). That is, suppose we are trying to fit the model  Y = \beta_{0} + \beta_{1}X_{1} + \...
58 views

### Getting the wrong sign on a coefficient in logistic regression?

I'm trying to make a logistic regression model explaining whether a law passed last year has affected my dependent variable. My most important variable (an indicator variable for whether the law was ...
13 views

### Testing for multi-collinearity after fitting a model with LassoCV from Sklearn in Python?

Is there a way to test for multi-collinearity, like VIF for example, after fitting a model with LassoCV from Sklearn in Python? https://scikit-learn.org/stable/modules/generated/sklearn.linear_model....
20 views

### Using a binary variable (present/absent) and its continuous counterpart (amount)

I am working on a banking-related logistic regression problem. Several of my variables are whether someone has an account of some sort, and there is a subsequent variable with the balance of the ...
43 views

### Convergence Error Using lifelines.CoxTimeVaryingFitter Python

I want to evaluate my Cox model using the lifelines package for a time varying covariate problem. However, when I use the lifelines.CoxTimeVaryingFitter I get a ...
11 views

### Does unbalanced data impact the Variance Inflation Factor?

I am interested in testing whether certain variables with a high correlation between them should be removed from the model. I was thinking of checking this out with VIF. I am working with a data set ...
43 views

### Working with systems with Perfect Multicollinearity

I am working with a time-series dataset that is based on demand-supply dynamics with several variables. THe sample data for one time period is: ...
17 views

### Working with systems with Perfect Multicollinearity

I am working with a time-series dataset that is based on demand-supply dynamics with variables: Production, Ending Stocks, Exports, Imports etc. I was working on a regression with this data and wanted ...
30 views

### Collinearity or not?

In a multiple regression analysis, if you have, in addition to host of other varibales, a dummy i.e. (0,1) coded gender variable with 0=male 1= female, and another dummy variable, say an occupational ...
8 views

### Addressing multi-collinearity issues with subset selection methods [duplicate]

I'm a long-time lurker and first-time poster to this forum... I am currently working my way through an Introduction to Statistical Learning, and I have a question regarding the algorithms presented ...
7 views

### Variable significance very sensitive to specification of non-correlated second variable

I´m doing research on a political science topic and my models leave me behind with a big questionmark at this point. I have a dataset containing 79 observations on a number of variables and trying and ...
18 views

### How does principal component regression help with multicollinearity?

In How to Deal with Multicollinearity?, the top comment in Aaron's answer says "These independent variables are now uncorrelated. Very low eigenvalues also indicate high degrees of ...
32 views

### Data with multicollinearity and p >> n

This is a csv file, the file is titled "res_final". The first line contains the names of the variables: ...
9 views

### Variance inflation factor vs condition number [duplicate]

We know the Variance inflation factor and condition number both help to measure multicollinearity. Which one should we use when?
32 views

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
40 views

### checking collinearity in a glm

I'm new to checking the VIF value for a glm model so I just want to make sure i"m understanding this correctly. I have 4 predictors for my count model and the model looks like this: ...
34 views

### Relationship between variables in a proposed model using linear regression?

I am new to linear regression and I am currently working on a linear regression problem - I have 8 features and one output. The features I am using seem unrelated to each other and I found an article (...
26 views

### Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
11 views

### What is the meaning of the regressor characteristic root?

As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
47 views

### Regression with highly negative correlated variable [duplicate]

I have a regression with 6 independent variables and one of them is highly negatively correlated with one of them. (-0.81 correlation) I was wondering if the regression is still valid or if it's ...
23 views

### Testing multi collinearity of ordinal variables in R

I want to test if there is multi collinearity in my dataset. This is made up of multiple ordinal independent variables. Am I correct that this should be done as shown below ...
21 views

### How does Ridge regression / Regularization help in selecting less or more important features? [duplicate]

Can someone please explain how regularization helps to shrink the " less important " features to zero ? As far as I know , Regularization only penalizes the weights of ALL the features to ...
17 views

### Multicollinearity in ECM model

In Error Correction Model I have proplem related to singular matrix. There are multicollinearity problem of such variables as exchange rate and interest rate. Because they are somehow stable over time,...
35 views

### What happens to the coefficients of Ridge and Lasso when you have perfect multicollinnearity?

So let's say we ran a Ridge or Lasso regression on $Y \sim X$, and get coefficient $\beta_X$. Now if we duplicate the $X$, and call it $Z$, and then run the same regression on: $Y \sim X + Z$. How ...
37 views

### Does multicollinearity produce wrong beta estimates?

This is the first time for me to ask a question here. I'm sorry that if I break any rule here. I have encountered a problem about the consequence of multicollinearity. During reading the explanation ...
41 views

### VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
59 views

### Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
61 views

### What is the difference between a confounder, collinearity, and interaction term?

These terms kind of confuse me because they all seem to imply a certain correlation. Confounder: influences dependent and independent variable Collinearity: to me just means correlation between ...
18 views

### Condition numbers, invertibility and multicollinearity

The following is an excerpt from Greene's Regression Analysis (Seventh Edition): a) What does it mean to be "difficult" to invert a matrix accurately? Shouldn't all matrixes be either ...
24 views

### Nested ANOVA: Correct use for colinearity problem?

We have neuronal data, that we want to analyse for effects of two factors that are hierarchically structured. The data: We have recorded neuronal data from a neuron, where the organism received visual ...
32 views

### Why does centering NOT cure multicollinearity?

In several posts, such as Is centering a valid solution for multicollinearity?, it states that centering doesn't solve multicollinearity because "it's a linear transformation." I just made ...
17 views

### Multicollinearity negating $\beta$

In Regression coefficients that flip sign after including other predictors, ars's answer states that "Basically, if your variables are positively correlated, then the coefficients will be ...
77 views

### How to identify which variables are collinear in a singular regression matrix? [duplicate]

I have a matrix on which I am performing a zero inflated regression model but it return an error indicating collinearity between some of the variables ...
23 views

### Should we remove a variable having low p value but high multicollinearity?

I have two regression models. The 2nd model is obtained by removing one variable from the first. The removed variable had high multicollinearity although very low p-value. (A variable RGDP was showing ...
59 views

### How does colinearity among the features impact classification problem in a) Logistic Regression b) Tree models like Decision Tree or Random Forest? [duplicate]

If I have 20 variables, should I do a pair-wise correlation check before building a classification model with Logistic Reg, Decision Tree or Random Forest ? Multicolinearity is a problem in regression....
51 views

### multicollinearity and predictive power

In wikipedia it says ...
34 views

### Geometric interpretation of multiple regression

I have two predictors $X_1$ and $X_2$, they are positively correlated. The response $Y$ are positively corrected with both $X_1$ and $X_2$, i.e. if we fit regression $Y=\beta_1 X_1$ and $Y=\beta_2 X_2$...
42 views

### why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
5 views

### Does changing reference variable of design matrix reduce multicollinearity?

It is stated here by Paul Allison that, if multicollinearity occurs within the design matrix of a categorical variable, multicollinearity can be reduced by setting the category with higher share of ...
44 views

### Collinearity in linear mixed models

Is collinearity of the independent fixed effects (covariates, so not the variables I'm interested in) a problem when you create a linear mixed model? Is there a good resource I can point to to support ...
55 views

### What's the difference between endogenous variable and multicollinearity?

Investopedia says that: Multicollinearity is a statistical concept where independent variables in a model are correlated. Also Investopedia says that: An endogenous variable is a variable in a ...
19 views

### Multicollinearity and Partial Dependence Questions

Assume I build a binary classification model to predict p(y=1) from {x1, x2, ... x10} For now, assume that model could be a GBM, RandomForest, or Logistic Regression. Also assume that all of the ...
29 views

### Working around multicollinearity

Given that I am far from an expert statistician, perhaps the community can inform me if my first thought to work around multicollinearity is going to result in questionable results. I have a model ...