Questions tagged [variance-inflation-factor]

Variance Inflation Factor (VIF) quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance (the square of the estimate's standard deviation) of an estimated regression coefficient is increased because of collinearity.

Filter by
Sorted by
Tagged with
1 vote
0 answers
17 views

Batchwise Variance Inflation Factor Calculation for Feature Selection

I am currently working on a research project that involves interpretation of coefficients from a fitted Cox Proportional Hazards model. Unfortunately, the features within my model exhibit high ...
user avatar
0 votes
0 answers
23 views

Multiple Regression - VIF

could anyone help me with my problem - I’m a beginner. When doing multiple regression and checking for multicollinearity, I have found 2 VIF’s of 17, however, I’ve done some reading and few say it is ...
user avatar
0 votes
0 answers
12 views

Average VIF after dummy variables [duplicate]

I have classified (label encoding) independent variables. To calculate VIF, I converted them to dummy variables. When I calculate VIF for dummy variables, there are VIF values for each class of an ...
user avatar
  • 1
0 votes
0 answers
17 views

Pearson correlation and VIF with categorical variables

All columns' values are class labels. For example: value "1" for feature1 is <50. Namely, all features were classified. When I apply Pearson correlation with continuous variables (before ...
user avatar
  • 1
1 vote
0 answers
13 views

How to Assess Multicollinearity for Independent Count and Rate Variables

How do I test an independent variable for multicollinearity if it comes from a Poisson or Negative Binomial distribution? A common approach for testing the multicollinearity of a model's independent ...
user avatar
  • 227
0 votes
0 answers
34 views

Multicollinearity: not by some measures (VIF, TOL), yes by others (Wi, Fi) - does this invalidate the regression model?

Data is from http://peopleanalytics-regression-book.org/data/sociological_data.csv After reading this into R as sociological_data, here is my code: ...
user avatar
0 votes
0 answers
28 views

Does the condition number depend on the sample size?

I am working with a dataset of 214 thousand firms and I am running a logistic regression. Not only my dependent variable (bOptingOut), but also some of my independent variables are binary (...
user avatar
0 votes
0 answers
27 views

Theoretical backing for VIF

I'm giving a lecture in machine learning class covering linear regression, and wanted to briefly touch up on the idea of multicollinearity since I come from a more traditional stats background. I was ...
user avatar
  • 1
2 votes
2 answers
198 views

PCA can make VIF worse? Why is it so often sold as a multicollinearity fix?

It's the conventional wisdom that a PCA transformation can cure multicollinearity. Putting this into practice on example data, I find myself confused. In the following case, applying PCA seems to have ...
user avatar
0 votes
0 answers
53 views

VIFs in Fixed-Effects-model

I know that R is providing some Warnings messages when you try to get your VIFs to test for multicollinearity. But why is that? Does someone know a good paper or the answer itself? CODE: ...
user avatar
0 votes
0 answers
97 views

Which threshold should I use for GVIF1/(2⋅df)? (Variance Inflation Factor)

I'm using the mtcars dataset in R, I used the car packages to estimate the VIF, but since I have factor variables I got the vif table with GVIF and GVIF1/(2⋅df) values, in another question Which ...
user avatar
  • 1
1 vote
1 answer
67 views

Is there any way to determine VIF of some variable that is included in the dataset that has so many variables?

I am new in statistics and need some help to determine the VIF value on all my variables/features in the dataset with a lot of variables. I have 98 variables with 76 observations and need to find VIF ...
user avatar
1 vote
0 answers
31 views

Can I use variance inflation factor (VIF) for logistic regression? [duplicate]

I have generated a glm model with 20 or so predictors. I have carried out stepwise regression(forward and backwards selection) to identify the important predictor variables. My final model has 7 ...
user avatar
  • 11
0 votes
0 answers
20 views

Is multicollinearity acceptable where one variable is the anomaly between, or sum of, other variables?

I'm using the variance inflation factor (VIF) to calculate multi-collinearity between the variables I'd like to include within a regression model. I was reading this article: https://...
user avatar
0 votes
0 answers
137 views

Running Variance Inflation Factor(VIF) iteratively for a large number of features taking a long time

I am trying to do logistic regression on a dataset with 1,500 features that are very multicolinear. I care about interpretability of coefficients so I am running a VIF calculation on all columns and ...
user avatar
0 votes
0 answers
123 views

Do you remove variables from a StatsModel logistic regression when you have a VIF of NaN?

I am building a logistic regression model to create a confusion matrix and decision tree. However, when checking for multicollinearity, I have a few variables that have a VIF of NaN. Do I remove them ...
user avatar
0 votes
0 answers
68 views

Appropriate test for independence of predictor variables in logistic regression

I want to test for independence between 7 categorical predictor variables in a logistic regression model. I believe the appropriate test would be a chi-square test, but I have 7 variables - is there a ...
user avatar
  • 19
1 vote
0 answers
50 views

Is that possible to rank the importance of independent variables with simple linear regression with standardized IVs

I run a multiple linear regression model with six independent variables using lm function in R (n=25). However, I found severe multicollinearity between my IVs (based on VIF values). The presence of ...
user avatar
0 votes
0 answers
83 views

calculate vif before or after standardization

hi I calculate vif before standardization results are: feature VIF 0 7.372450 1 2.116071 2 1.283643 3 1.268624 4 1.730986 5 5.442314 6 1.266718 7 3.183669 after ...
user avatar
2 votes
1 answer
56 views

How to show VIF

The variance of the $j$th element of the OLS estimator is given by $$\operatorname{Var}\left(\hat{\beta}_{j}\right)=\sigma^{2}\left(X_{j}^{T} M_{-j} X_{j}\right)^{-1}$$ where $X_j$ is the column of ...
user avatar
  • 21
1 vote
1 answer
131 views

When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
user avatar
  • 727
1 vote
1 answer
54 views

OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
user avatar
1 vote
1 answer
24 views

In a GLMM model 2 predictors (covariates) correlate positively but affect the response with opposite relationships. Is it possibile?

I am checking the results of my GLMM model ...
user avatar
  • 31
1 vote
0 answers
105 views

VIF (multicollinearity), Breusch-Godfrey (autocorrelation), Breusch-Pagan (heteroskedasticity) for Linear Regression

We are conducting linear regression. We performed first the Variance Inflation Factor to check for multicollinearity and we dropped the independent variables below 10 So are ALL of our independent ...
user avatar
0 votes
0 answers
93 views

Is the formula of VIF correct compared with partial correlation (variance)

For a variable $x = (x_1,\cdots,x_n)$ and its sample matrix $X,$ we know the partial correlation of $x_i,x_j$ is the correlation of their residuals respect to the ...
user avatar
  • 1,017
0 votes
0 answers
76 views

Which machine learning model is reliable when my data face multicollinearity issue?

I have 27 features and I'm trying to predict continuous values. When I calculated the VIF (VarianceInflation Factors), only 8 features are less than 10 and the remaining features range from 10 to 250. ...
user avatar
2 votes
2 answers
514 views

Why multicollinearity increases with country fixed effects in linear model in r

I'm playing with some multiple linear regression models in r. After I run a regression, I use vif() to see if there is multicollinearity between my predictors. For ...
user avatar
  • 331
0 votes
0 answers
105 views

When can we not drop an independent variable that has a high VIF?

In a linear regression context, and we observe that some independent variable can be approximately written as a linear combination of a set of other independent variables (e.g., with $R^2 > 0.95 \...
user avatar
0 votes
0 answers
29 views

Is looking at the correlation table between predictors an insufficient test for multicollinearity?

In some of these data science analyses that I've seen posted on the web, e.g., https://towardsdatascience.com/linear-regression-on-boston-housing-dataset-f409b7e4a155, they "test" for ...
user avatar
  • 927
1 vote
0 answers
75 views

Using VIF, Interaction Effects, polynomial associations for feature selection in multiple linear regression

Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when ...
user avatar
0 votes
0 answers
62 views

Sample or Population Standard Deviation (SD) While Calculating SD Regarding All Variables' Variance Inflation Factor Values

What Was I Trying to Do? I was calculating the Variance Inflation Factors (VIFs) of all variables in a data set. Then, I was interested to calculate the mean and standard deviation (SD) of VIFs of all ...
user avatar
2 votes
1 answer
234 views

low VIF but having correlation in multiple regression?

...
user avatar
  • 37
0 votes
0 answers
227 views

Negative GVIF values in multinomial logistic regression in R

I'm trying to check the vif for a multinomial logistic regression with categorical as well as continuous variables as explanatory variables. I'm using the function vif() from car package in R. However ...
user avatar
2 votes
0 answers
49 views

Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
user avatar
  • 435
1 vote
1 answer
58 views

Variance Inflation Factor - can I make this assumption?

Let's assume I have a dataset with two numeric columns and I am assessing VIF for each variable: [x1, x2] Now let's assume that the calculation shows that variable <...
user avatar
  • 383
1 vote
1 answer
212 views

Multiple regression with dummy variables: identical VIF, Tolerance and Standard Error

Im fairly new to stats and regression but trying to learn and I've come across something that doesn't seem right to me. I have used dummy variables to run a multiple regression model to predict the ...
user avatar
0 votes
0 answers
21 views

Modeling the association between multiple correlated variables

I have a set of 10 variables all based around inventory optimization. Many of the variables are highly correlated. The ask is to determine the magnitude of the increase in one variable based on the ...
user avatar
3 votes
0 answers
73 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
user avatar
  • 51
1 vote
1 answer
218 views

VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
user avatar
  • 296
3 votes
1 answer
744 views

Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
user avatar
  • 296
2 votes
1 answer
2k views

calculating variance inflation factor for logistic regression using statsmodels (or python)?

I am making a logistic regression model using Statsmodels while following the book "Discovering statistics using R" by Andy Field, Jeremy Miles, and Zoë Field . While following along the ...
user avatar
  • 123
1 vote
1 answer
285 views

why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
user avatar
0 votes
1 answer
379 views

Variance Inflation Factors for a glm with clustered standard errors

I am using the glm.cluster function in R package miceadds and I would like to calculate the variance inflation factors (VIF), much as the vif function in R package car does. If I try to use car::vif I ...
user avatar
1 vote
0 answers
15 views

Logistic or linear regression in VIF function to assess for multicollinearity?

I am running a logistic regression but I want to test for multicollinearity first. Can I put the logistic regression in the VIF function or do I first have to make a linear regression put that in the ...
user avatar
3 votes
1 answer
752 views

Is it advisable to impute missing values and scale features before computing the Variance Inflation Factor (VIF)?

As far as scaling, Wikipedia says: Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF). ...
user avatar
  • 259
1 vote
1 answer
2k views

How is VIF calculated for dummy variables?

I have a logistic regression model with 11 explanatory variables, 5 of which are dummy variables, when I use vif() function from library car in R, it gives me a VIF value for each of them. As far as I ...
user avatar
  • 153
0 votes
1 answer
68 views

High Variance inflation factor for all variables

As you can see there vif for all variables is too high...My interpretation is that all variables are highly correlated to each other. How can I further investigate on this? Or how can this be further ...
user avatar
  • 31
1 vote
0 answers
81 views

Variance of Linear Regression Coefficients in Terms of VIF

I am trying to understand how to rewrite the variance of regression coefficients in terms of the variance inflation factor. To be specific If we have a regression with design matrix X, I know that we ...
user avatar
  • 111
5 votes
1 answer
1k views

Why high correlation coefficient doesn't guarantee high VIF?

I know that there are similar questions about this topic, but my question is not really which metric should we depend on but why they are not equivalent? There is a saying in the book Basic ...
user avatar
  • 493
8 votes
2 answers
879 views

Isn't it futile to try to fix multicollinearity?

Most of the advice on how to deal with multicolinear predictors tells you to eliminate them before fitting your model, using some criterium like VIF (Variance Inflation Factor). If I understand it ...
user avatar