# Questions tagged [variance-inflation-factor]

Variance Inflation Factor (VIF) quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance (the square of the estimate's standard deviation) of an estimated regression coefficient is increased because of collinearity.

171 questions
Filter by
Sorted by
Tagged with
19 views

### Multicollinearity: not by some measures (VIF, TOL), yes by others (Wi, Fi) - does this invalidate the regression model?

Data is from http://peopleanalytics-regression-book.org/data/sociological_data.csv After reading this into R as sociological_data, here is my code: ...
• 541
23 views

### Does the condition number depend on the sample size?

I am working with a dataset of 214 thousand firms and I am running a logistic regression. Not only my dependent variable (bOptingOut), but also some of my independent variables are binary (...
27 views

### Theoretical backing for VIF

I'm giving a lecture in machine learning class covering linear regression, and wanted to briefly touch up on the idea of multicollinearity since I come from a more traditional stats background. I was ...
• 1
171 views

### PCA can make VIF worse? Why is it so often sold as a multicollinearity fix?

It's the conventional wisdom that a PCA transformation can cure multicollinearity. Putting this into practice on example data, I find myself confused. In the following case, applying PCA seems to have ...
41 views

### VIFs in Fixed-Effects-model

I know that R is providing some Warnings messages when you try to get your VIFs to test for multicollinearity. But why is that? Does someone know a good paper or the answer itself? CODE: ...
65 views

### Which threshold should I use for GVIF1/(2⋅df)? (Variance Inflation Factor)

I'm using the mtcars dataset in R, I used the car packages to estimate the VIF, but since I have factor variables I got the vif table with GVIF and GVIF1/(2⋅df) values, in another question Which ...
1 vote
29 views

### Is there any way to determine VIF of some variable that is included in the dataset that has so many variables?

I am new in statistics and need some help to determine the VIF value on all my variables/features in the dataset with a lot of variables. I have 98 variables with 76 observations and need to find VIF ...
• 11
1 vote
29 views

### Can I use variance inflation factor (VIF) for logistic regression? [duplicate]

I have generated a glm model with 20 or so predictors. I have carried out stepwise regression(forward and backwards selection) to identify the important predictor variables. My final model has 7 ...
• 11
18 views

### Is multicollinearity acceptable where one variable is the anomaly between, or sum of, other variables?

I'm using the variance inflation factor (VIF) to calculate multi-collinearity between the variables I'd like to include within a regression model. I was reading this article: https://...
78 views

### Running Variance Inflation Factor(VIF) iteratively for a large number of features taking a long time

I am trying to do logistic regression on a dataset with 1,500 features that are very multicolinear. I care about interpretability of coefficients so I am running a VIF calculation on all columns and ...
97 views

### Do you remove variables from a StatsModel logistic regression when you have a VIF of NaN?

I am building a logistic regression model to create a confusion matrix and decision tree. However, when checking for multicollinearity, I have a few variables that have a VIF of NaN. Do I remove them ...
47 views

### Appropriate test for independence of predictor variables in logistic regression

I want to test for independence between 7 categorical predictor variables in a logistic regression model. I believe the appropriate test would be a chi-square test, but I have 7 variables - is there a ...
• 19
1 vote
50 views

### Is that possible to rank the importance of independent variables with simple linear regression with standardized IVs

I run a multiple linear regression model with six independent variables using lm function in R (n=25). However, I found severe multicollinearity between my IVs (based on VIF values). The presence of ...
67 views

### calculate vif before or after standardization

hi I calculate vif before standardization results are: feature VIF 0 7.372450 1 2.116071 2 1.283643 3 1.268624 4 1.730986 5 5.442314 6 1.266718 7 3.183669 after ...
56 views

### How to show VIF

The variance of the $j$th element of the OLS estimator is given by $$\operatorname{Var}\left(\hat{\beta}_{j}\right)=\sigma^{2}\left(X_{j}^{T} M_{-j} X_{j}\right)^{-1}$$ where $X_j$ is the column of ...
• 21
1 vote
86 views

### When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
• 707
1 vote
46 views

### OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
• 541
1 vote
22 views

### In a GLMM model 2 predictors (covariates) correlate positively but affect the response with opposite relationships. Is it possibile?

I am checking the results of my GLMM model ...
• 31
1 vote
91 views

### VIF (multicollinearity), Breusch-Godfrey (autocorrelation), Breusch-Pagan (heteroskedasticity) for Linear Regression

We are conducting linear regression. We performed first the Variance Inflation Factor to check for multicollinearity and we dropped the independent variables below 10 So are ALL of our independent ...
77 views

### Is the formula of VIF correct compared with partial correlation (variance)

For a variable $x = (x_1,\cdots,x_n)$ and its sample matrix $X,$ we know the partial correlation of $x_i,x_j$ is the correlation of their residuals respect to the ...
• 885
68 views

### Which machine learning model is reliable when my data face multicollinearity issue?

I have 27 features and I'm trying to predict continuous values. When I calculated the VIF (VarianceInflation Factors), only 8 features are less than 10 and the remaining features range from 10 to 250. ...
417 views

### Why multicollinearity increases with country fixed effects in linear model in r

I'm playing with some multiple linear regression models in r. After I run a regression, I use vif() to see if there is multicollinearity between my predictors. For ...
• 331
99 views

### When can we not drop an independent variable that has a high VIF?

In a linear regression context, and we observe that some independent variable can be approximately written as a linear combination of a set of other independent variables (e.g., with \$R^2 > 0.95 \...
26 views

### Is looking at the correlation table between predictors an insufficient test for multicollinearity?

In some of these data science analyses that I've seen posted on the web, e.g., https://towardsdatascience.com/linear-regression-on-boston-housing-dataset-f409b7e4a155, they "test" for ...
• 877
1 vote
70 views

### Using VIF, Interaction Effects, polynomial associations for feature selection in multiple linear regression

Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when ...
• 205
56 views

### Sample or Population Standard Deviation (SD) While Calculating SD Regarding All Variables' Variance Inflation Factor Values

What Was I Trying to Do? I was calculating the Variance Inflation Factors (VIFs) of all variables in a data set. Then, I was interested to calculate the mean and standard deviation (SD) of VIFs of all ...
1 vote
187 views

...
• 27
204 views

### Negative GVIF values in multinomial logistic regression in R

I'm trying to check the vif for a multinomial logistic regression with categorical as well as continuous variables as explanatory variables. I'm using the function vif() from car package in R. However ...
49 views

### Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
• 383
1 vote
47 views

### Variance Inflation Factor - can I make this assumption?

Let's assume I have a dataset with two numeric columns and I am assessing VIF for each variable: [x1, x2] Now let's assume that the calculation shows that variable <...
• 383
1 vote
192 views

### Multiple regression with dummy variables: identical VIF, Tolerance and Standard Error

Im fairly new to stats and regression but trying to learn and I've come across something that doesn't seem right to me. I have used dummy variables to run a multiple regression model to predict the ...
21 views

### Modeling the association between multiple correlated variables

I have a set of 10 variables all based around inventory optimization. Many of the variables are highly correlated. The ask is to determine the magnitude of the increase in one variable based on the ...
63 views

### Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
• 51
1 vote
190 views

### VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
• 306
665 views

### Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
• 306
1 vote
2k views

### calculating variance inflation factor for logistic regression using statsmodels (or python)?

I am making a logistic regression model using Statsmodels while following the book "Discovering statistics using R" by Andy Field, Jeremy Miles, and Zoë Field . While following along the ...
• 113
1 vote
243 views

### why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
355 views

### Variance Inflation Factors for a glm with clustered standard errors

I am using the glm.cluster function in R package miceadds and I would like to calculate the variance inflation factors (VIF), much as the vif function in R package car does. If I try to use car::vif I ...
1 vote
15 views

### Logistic or linear regression in VIF function to assess for multicollinearity?

I am running a logistic regression but I want to test for multicollinearity first. Can I put the logistic regression in the VIF function or do I first have to make a linear regression put that in the ...
664 views

### Is it advisable to impute missing values and scale features before computing the Variance Inflation Factor (VIF)?

As far as scaling, Wikipedia says: Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF). ...
• 259
1 vote
2k views

### How is VIF calculated for dummy variables?

I have a logistic regression model with 11 explanatory variables, 5 of which are dummy variables, when I use vif() function from library car in R, it gives me a VIF value for each of them. As far as I ...
• 163
62 views

### High Variance inflation factor for all variables

As you can see there vif for all variables is too high...My interpretation is that all variables are highly correlated to each other. How can I further investigate on this? Or how can this be further ...
• 31
1 vote
78 views

### Variance of Linear Regression Coefficients in Terms of VIF

I am trying to understand how to rewrite the variance of regression coefficients in terms of the variance inflation factor. To be specific If we have a regression with design matrix X, I know that we ...
• 111
948 views

### Why high correlation coefficient doesn't guarantee high VIF?

I know that there are similar questions about this topic, but my question is not really which metric should we depend on but why they are not equivalent? There is a saying in the book Basic ...
• 493
879 views

### Isn't it futile to try to fix multicollinearity?

Most of the advice on how to deal with multicolinear predictors tells you to eliminate them before fitting your model, using some criterium like VIF (Variance Inflation Factor). If I understand it ...
77 views

### What to do in regression when an important continous covariate is correlated with the grouping covariate?

Hoping I can explain my problem clearly enough. My Masters thesis is looking at the impacts of elephant on woody vegetation in relation to distance from waterholes and vegetation type. Vegetation ...
• 143
26 views

### Is the arcsin trasformation useful to reduce multicollinearity?

I would like to use a GLM but, testing multicollinearity using the VIF parameter, I obtained higher correlation among independent variables (expressed as percentages). Is the arcsin trasformation ...
1k views

### Collinearity in factor dummies

I am running a linear regression on the house prices dataset from Kaggle and I was trying to see if I can find collinearity between some variables in order to improve my prediction. To this purpose ...
• 113
1k views

### how to calculate VIF in logistic regression?

No book told me that... Using McFadden’s Pseudo-R2 ? OR do traditional linear regression to get VIF?
• 43
1 vote