Questions tagged [variance-inflation-factor]

Variance Inflation Factor (VIF) quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance (the square of the estimate's standard deviation) of an estimated regression coefficient is increased because of collinearity.

Filter by
Sorted by
Tagged with
2 votes
2 answers
27 views

Checking for significant associations between a mixed set of predictors prior to running a conditional random forest model

I am running a conditional random forest model using the party package in R, with the goal of quantifying variable importance (permutation importance) for 29 predictor variables. My response variable ...
0 votes
0 answers
18 views

identifying the multicollinearity when Intercept has a very high variance

I applied the multinomial logistic regression and use the following function colldiag to calculate the Multicolinearity. It seems bottom line (index=39) shows ...
  • 13
2 votes
2 answers
114 views

What to do when every VIF value is infinity

I know that VIF values have no upper limit, and that anything over 10 is usually bad news if you are trying to avoid multicollinearity especially for regression models such as multiple logistic ...
1 vote
0 answers
35 views

VIF - but with a categorical response variable?

I've seen a tonne of tutorials online for managing categorical variables in a VIF test by making binary dummy variables, but with a variable with 3 or more categories this means creating two or more ...
1 vote
0 answers
29 views

Batchwise Variance Inflation Factor Calculation for Feature Selection

I am currently working on a research project that involves interpretation of coefficients from a fitted Cox Proportional Hazards model. Unfortunately, the features within my model exhibit high ...
0 votes
0 answers
31 views

Multiple Regression - VIF

could anyone help me with my problem - I’m a beginner. When doing multiple regression and checking for multicollinearity, I have found 2 VIF’s of 17, however, I’ve done some reading and few say it is ...
0 votes
0 answers
12 views

Average VIF after dummy variables [duplicate]

I have classified (label encoding) independent variables. To calculate VIF, I converted them to dummy variables. When I calculate VIF for dummy variables, there are VIF values for each class of an ...
  • 1
0 votes
0 answers
27 views

Pearson correlation and VIF with categorical variables

All columns' values are class labels. For example: value "1" for feature1 is <50. Namely, all features were classified. When I apply Pearson correlation with continuous variables (before ...
  • 1
1 vote
0 answers
17 views

How to Assess Multicollinearity for Independent Count and Rate Variables

How do I test an independent variable for multicollinearity if it comes from a Poisson or Negative Binomial distribution? A common approach for testing the multicollinearity of a model's independent ...
  • 227
0 votes
0 answers
52 views

Multicollinearity: not by some measures (VIF, TOL), yes by others (Wi, Fi) - does this invalidate the regression model?

Data is from http://peopleanalytics-regression-book.org/data/sociological_data.csv After reading this into R as sociological_data, here is my code: ...
0 votes
0 answers
39 views

Does the condition number depend on the sample size?

I am working with a dataset of 214 thousand firms and I am running a logistic regression. Not only my dependent variable (bOptingOut), but also some of my independent variables are binary (...
0 votes
0 answers
29 views

Theoretical backing for VIF

I'm giving a lecture in machine learning class covering linear regression, and wanted to briefly touch up on the idea of multicollinearity since I come from a more traditional stats background. I was ...
  • 1
2 votes
2 answers
229 views

PCA can make VIF worse? Why is it so often sold as a multicollinearity fix?

It's the conventional wisdom that a PCA transformation can cure multicollinearity. Putting this into practice on example data, I find myself confused. In the following case, applying PCA seems to have ...
0 votes
0 answers
89 views

VIFs in Fixed-Effects-model

I know that R is providing some Warnings messages when you try to get your VIFs to test for multicollinearity. But why is that? Does someone know a good paper or the answer itself? CODE: ...
0 votes
0 answers
151 views

Which threshold should I use for GVIF1/(2⋅df)? (Variance Inflation Factor)

I'm using the mtcars dataset in R, I used the car packages to estimate the VIF, but since I have factor variables I got the vif table with GVIF and GVIF1/(2⋅df) values, in another question Which ...
  • 1
1 vote
1 answer
172 views

Is there any way to determine VIF of some variable that is included in the dataset that has so many variables?

I am new in statistics and need some help to determine the VIF value on all my variables/features in the dataset with a lot of variables. I have 98 variables with 76 observations and need to find VIF ...
1 vote
0 answers
31 views

Can I use variance inflation factor (VIF) for logistic regression? [duplicate]

I have generated a glm model with 20 or so predictors. I have carried out stepwise regression(forward and backwards selection) to identify the important predictor variables. My final model has 7 ...
  • 11
0 votes
0 answers
39 views

Is multicollinearity acceptable where one variable is the anomaly between, or sum of, other variables?

I'm using the variance inflation factor (VIF) to calculate multi-collinearity between the variables I'd like to include within a regression model. I was reading this article: https://...
0 votes
0 answers
221 views

Running Variance Inflation Factor(VIF) iteratively for a large number of features taking a long time

I am trying to do logistic regression on a dataset with 1,500 features that are very multicolinear. I care about interpretability of coefficients so I am running a VIF calculation on all columns and ...
0 votes
0 answers
169 views

Do you remove variables from a StatsModel logistic regression when you have a VIF of NaN?

I am building a logistic regression model to create a confusion matrix and decision tree. However, when checking for multicollinearity, I have a few variables that have a VIF of NaN. Do I remove them ...
0 votes
0 answers
90 views

Appropriate test for independence of predictor variables in logistic regression

I want to test for independence between 7 categorical predictor variables in a logistic regression model. I believe the appropriate test would be a chi-square test, but I have 7 variables - is there a ...
  • 19
1 vote
0 answers
58 views

Is that possible to rank the importance of independent variables with simple linear regression with standardized IVs

I run a multiple linear regression model with six independent variables using lm function in R (n=25). However, I found severe multicollinearity between my IVs (based on VIF values). The presence of ...
0 votes
0 answers
119 views

calculate vif before or after standardization

hi I calculate vif before standardization results are: feature VIF 0 7.372450 1 2.116071 2 1.283643 3 1.268624 4 1.730986 5 5.442314 6 1.266718 7 3.183669 after ...
2 votes
1 answer
57 views

How to show VIF

The variance of the $j$th element of the OLS estimator is given by $$\operatorname{Var}\left(\hat{\beta}_{j}\right)=\sigma^{2}\left(X_{j}^{T} M_{-j} X_{j}\right)^{-1}$$ where $X_j$ is the column of ...
  • 21
1 vote
1 answer
224 views

When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
  • 727
1 vote
1 answer
72 views

OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
1 vote
1 answer
26 views

In a GLMM model 2 predictors (covariates) correlate positively but affect the response with opposite relationships. Is it possibile?

I am checking the results of my GLMM model ...
  • 31
0 votes
0 answers
95 views

Which machine learning model is reliable when my data face multicollinearity issue?

I have 27 features and I'm trying to predict continuous values. When I calculated the VIF (VarianceInflation Factors), only 8 features are less than 10 and the remaining features range from 10 to 250. ...
3 votes
2 answers
635 views

Why multicollinearity increases with country fixed effects in a linear model in r

I'm playing with some multiple linear regression models in r. After I run a regression, I use vif() to see if there is multicollinearity between my predictors. For ...
  • 341
0 votes
0 answers
119 views

When can we not drop an independent variable that has a high VIF?

In a linear regression context, and we observe that some independent variable can be approximately written as a linear combination of a set of other independent variables (e.g., with $R^2 > 0.95 \...
0 votes
0 answers
36 views

Is looking at the correlation table between predictors an insufficient test for multicollinearity?

In some of these data science analyses that I've seen posted on the web, e.g., https://towardsdatascience.com/linear-regression-on-boston-housing-dataset-f409b7e4a155, they "test" for ...
  • 1,047
1 vote
0 answers
85 views

Using VIF, Interaction Effects, polynomial associations for feature selection in multiple linear regression

Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when ...
0 votes
0 answers
65 views

Sample or Population Standard Deviation (SD) While Calculating SD Regarding All Variables' Variance Inflation Factor Values

What Was I Trying to Do? I was calculating the Variance Inflation Factors (VIFs) of all variables in a data set. Then, I was interested to calculate the mean and standard deviation (SD) of VIFs of all ...
2 votes
1 answer
347 views

low VIF but having correlation in multiple regression?

...
  • 37
0 votes
0 answers
266 views

Negative GVIF values in multinomial logistic regression in R

I'm trying to check the vif for a multinomial logistic regression with categorical as well as continuous variables as explanatory variables. I'm using the function vif() from car package in R. However ...
2 votes
0 answers
50 views

Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
  • 507
1 vote
1 answer
63 views

Variance Inflation Factor - can I make this assumption?

Let's assume I have a dataset with two numeric columns and I am assessing VIF for each variable: [x1, x2] Now let's assume that the calculation shows that variable <...
  • 393
1 vote
1 answer
257 views

Multiple regression with dummy variables: identical VIF, Tolerance and Standard Error

Im fairly new to stats and regression but trying to learn and I've come across something that doesn't seem right to me. I have used dummy variables to run a multiple regression model to predict the ...
0 votes
0 answers
23 views

Modeling the association between multiple correlated variables

I have a set of 10 variables all based around inventory optimization. Many of the variables are highly correlated. The ask is to determine the magnitude of the increase in one variable based on the ...
3 votes
0 answers
88 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
  • 51
1 vote
1 answer
261 views

VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
  • 296
3 votes
1 answer
845 views

Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
  • 296
2 votes
1 answer
3k views

calculating variance inflation factor for logistic regression using statsmodels (or python)?

I am making a logistic regression model using Statsmodels while following the book "Discovering statistics using R" by Andy Field, Jeremy Miles, and Zoë Field . While following along the ...
  • 123
1 vote
1 answer
341 views

why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
0 votes
1 answer
418 views

Variance Inflation Factors for a glm with clustered standard errors

I am using the glm.cluster function in R package miceadds and I would like to calculate the variance inflation factors (VIF), much as the vif function in R package car does. If I try to use car::vif I ...
1 vote
0 answers
16 views

Logistic or linear regression in VIF function to assess for multicollinearity?

I am running a logistic regression but I want to test for multicollinearity first. Can I put the logistic regression in the VIF function or do I first have to make a linear regression put that in the ...
3 votes
1 answer
899 views

Is it advisable to impute missing values and scale features before computing the Variance Inflation Factor (VIF)?

As far as scaling, Wikipedia says: Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF). ...
  • 269
1 vote
1 answer
3k views

How is VIF calculated for dummy variables?

I have a logistic regression model with 11 explanatory variables, 5 of which are dummy variables, when I use vif() function from library car in R, it gives me a VIF value for each of them. As far as I ...
  • 153
0 votes
1 answer
81 views

High Variance inflation factor for all variables

As you can see there vif for all variables is too high...My interpretation is that all variables are highly correlated to each other. How can I further investigate on this? Or how can this be further ...
  • 31
1 vote
0 answers
86 views

Variance of Linear Regression Coefficients in Terms of VIF

I am trying to understand how to rewrite the variance of regression coefficients in terms of the variance inflation factor. To be specific If we have a regression with design matrix X, I know that we ...
  • 111