Questions tagged [variance-inflation-factor]

Variance Inflation Factor (VIF) quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance (the square of the estimate's standard deviation) of an estimated regression coefficient is increased because of collinearity.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

Multicollinearity: not by some measures (VIF, TOL), yes by others (Wi, Fi) - does this invalidate the regression model?

Data is from http://peopleanalytics-regression-book.org/data/sociological_data.csv After reading this into R as sociological_data, here is my code: ...
user avatar
0 votes
0 answers
23 views

Does the condition number depend on the sample size?

I am working with a dataset of 214 thousand firms and I am running a logistic regression. Not only my dependent variable (bOptingOut), but also some of my independent variables are binary (...
user avatar
0 votes
0 answers
27 views

Theoretical backing for VIF

I'm giving a lecture in machine learning class covering linear regression, and wanted to briefly touch up on the idea of multicollinearity since I come from a more traditional stats background. I was ...
user avatar
  • 1
2 votes
2 answers
171 views

PCA can make VIF worse? Why is it so often sold as a multicollinearity fix?

It's the conventional wisdom that a PCA transformation can cure multicollinearity. Putting this into practice on example data, I find myself confused. In the following case, applying PCA seems to have ...
user avatar
0 votes
0 answers
41 views

VIFs in Fixed-Effects-model

I know that R is providing some Warnings messages when you try to get your VIFs to test for multicollinearity. But why is that? Does someone know a good paper or the answer itself? CODE: ...
user avatar
0 votes
0 answers
65 views

Which threshold should I use for GVIF1/(2⋅df)? (Variance Inflation Factor)

I'm using the mtcars dataset in R, I used the car packages to estimate the VIF, but since I have factor variables I got the vif table with GVIF and GVIF1/(2⋅df) values, in another question Which ...
user avatar
  • 1
1 vote
1 answer
29 views

Is there any way to determine VIF of some variable that is included in the dataset that has so many variables?

I am new in statistics and need some help to determine the VIF value on all my variables/features in the dataset with a lot of variables. I have 98 variables with 76 observations and need to find VIF ...
user avatar
1 vote
0 answers
29 views

Can I use variance inflation factor (VIF) for logistic regression? [duplicate]

I have generated a glm model with 20 or so predictors. I have carried out stepwise regression(forward and backwards selection) to identify the important predictor variables. My final model has 7 ...
user avatar
  • 11
0 votes
0 answers
18 views

Is multicollinearity acceptable where one variable is the anomaly between, or sum of, other variables?

I'm using the variance inflation factor (VIF) to calculate multi-collinearity between the variables I'd like to include within a regression model. I was reading this article: https://...
user avatar
0 votes
0 answers
78 views

Running Variance Inflation Factor(VIF) iteratively for a large number of features taking a long time

I am trying to do logistic regression on a dataset with 1,500 features that are very multicolinear. I care about interpretability of coefficients so I am running a VIF calculation on all columns and ...
user avatar
0 votes
0 answers
97 views

Do you remove variables from a StatsModel logistic regression when you have a VIF of NaN?

I am building a logistic regression model to create a confusion matrix and decision tree. However, when checking for multicollinearity, I have a few variables that have a VIF of NaN. Do I remove them ...
user avatar
0 votes
0 answers
47 views

Appropriate test for independence of predictor variables in logistic regression

I want to test for independence between 7 categorical predictor variables in a logistic regression model. I believe the appropriate test would be a chi-square test, but I have 7 variables - is there a ...
user avatar
  • 19
1 vote
0 answers
50 views

Is that possible to rank the importance of independent variables with simple linear regression with standardized IVs

I run a multiple linear regression model with six independent variables using lm function in R (n=25). However, I found severe multicollinearity between my IVs (based on VIF values). The presence of ...
user avatar
0 votes
0 answers
67 views

calculate vif before or after standardization

hi I calculate vif before standardization results are: feature VIF 0 7.372450 1 2.116071 2 1.283643 3 1.268624 4 1.730986 5 5.442314 6 1.266718 7 3.183669 after ...
user avatar
2 votes
1 answer
56 views

How to show VIF

The variance of the $j$th element of the OLS estimator is given by $$\operatorname{Var}\left(\hat{\beta}_{j}\right)=\sigma^{2}\left(X_{j}^{T} M_{-j} X_{j}\right)^{-1}$$ where $X_j$ is the column of ...
user avatar
  • 21
1 vote
1 answer
86 views

When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
user avatar
  • 707
1 vote
1 answer
46 views

OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
user avatar
1 vote
1 answer
22 views

In a GLMM model 2 predictors (covariates) correlate positively but affect the response with opposite relationships. Is it possibile?

I am checking the results of my GLMM model ...
user avatar
  • 31
1 vote
0 answers
91 views

VIF (multicollinearity), Breusch-Godfrey (autocorrelation), Breusch-Pagan (heteroskedasticity) for Linear Regression

We are conducting linear regression. We performed first the Variance Inflation Factor to check for multicollinearity and we dropped the independent variables below 10 So are ALL of our independent ...
user avatar
0 votes
0 answers
77 views

Is the formula of VIF correct compared with partial correlation (variance)

For a variable $x = (x_1,\cdots,x_n)$ and its sample matrix $X,$ we know the partial correlation of $x_i,x_j$ is the correlation of their residuals respect to the ...
user avatar
0 votes
0 answers
68 views

Which machine learning model is reliable when my data face multicollinearity issue?

I have 27 features and I'm trying to predict continuous values. When I calculated the VIF (VarianceInflation Factors), only 8 features are less than 10 and the remaining features range from 10 to 250. ...
user avatar
2 votes
2 answers
417 views

Why multicollinearity increases with country fixed effects in linear model in r

I'm playing with some multiple linear regression models in r. After I run a regression, I use vif() to see if there is multicollinearity between my predictors. For ...
user avatar
  • 331
0 votes
0 answers
99 views

When can we not drop an independent variable that has a high VIF?

In a linear regression context, and we observe that some independent variable can be approximately written as a linear combination of a set of other independent variables (e.g., with $R^2 > 0.95 \...
user avatar
0 votes
0 answers
26 views

Is looking at the correlation table between predictors an insufficient test for multicollinearity?

In some of these data science analyses that I've seen posted on the web, e.g., https://towardsdatascience.com/linear-regression-on-boston-housing-dataset-f409b7e4a155, they "test" for ...
user avatar
  • 877
1 vote
0 answers
70 views

Using VIF, Interaction Effects, polynomial associations for feature selection in multiple linear regression

Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when ...
user avatar
0 votes
0 answers
56 views

Sample or Population Standard Deviation (SD) While Calculating SD Regarding All Variables' Variance Inflation Factor Values

What Was I Trying to Do? I was calculating the Variance Inflation Factors (VIFs) of all variables in a data set. Then, I was interested to calculate the mean and standard deviation (SD) of VIFs of all ...
user avatar
1 vote
1 answer
187 views

low VIF but having correlation in multiple regression?

...
user avatar
  • 27
0 votes
0 answers
204 views

Negative GVIF values in multinomial logistic regression in R

I'm trying to check the vif for a multinomial logistic regression with categorical as well as continuous variables as explanatory variables. I'm using the function vif() from car package in R. However ...
user avatar
2 votes
0 answers
49 views

Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
user avatar
  • 383
1 vote
1 answer
47 views

Variance Inflation Factor - can I make this assumption?

Let's assume I have a dataset with two numeric columns and I am assessing VIF for each variable: [x1, x2] Now let's assume that the calculation shows that variable <...
user avatar
  • 383
1 vote
1 answer
192 views

Multiple regression with dummy variables: identical VIF, Tolerance and Standard Error

Im fairly new to stats and regression but trying to learn and I've come across something that doesn't seem right to me. I have used dummy variables to run a multiple regression model to predict the ...
user avatar
0 votes
0 answers
21 views

Modeling the association between multiple correlated variables

I have a set of 10 variables all based around inventory optimization. Many of the variables are highly correlated. The ask is to determine the magnitude of the increase in one variable based on the ...
user avatar
3 votes
0 answers
63 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
user avatar
  • 51
1 vote
1 answer
190 views

VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
user avatar
  • 306
4 votes
1 answer
665 views

Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
user avatar
  • 306
1 vote
1 answer
2k views

calculating variance inflation factor for logistic regression using statsmodels (or python)?

I am making a logistic regression model using Statsmodels while following the book "Discovering statistics using R" by Andy Field, Jeremy Miles, and Zoë Field . While following along the ...
user avatar
  • 113
1 vote
1 answer
243 views

why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
user avatar
0 votes
1 answer
355 views

Variance Inflation Factors for a glm with clustered standard errors

I am using the glm.cluster function in R package miceadds and I would like to calculate the variance inflation factors (VIF), much as the vif function in R package car does. If I try to use car::vif I ...
user avatar
1 vote
0 answers
15 views

Logistic or linear regression in VIF function to assess for multicollinearity?

I am running a logistic regression but I want to test for multicollinearity first. Can I put the logistic regression in the VIF function or do I first have to make a linear regression put that in the ...
user avatar
3 votes
1 answer
664 views

Is it advisable to impute missing values and scale features before computing the Variance Inflation Factor (VIF)?

As far as scaling, Wikipedia says: Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF). ...
user avatar
  • 259
1 vote
1 answer
2k views

How is VIF calculated for dummy variables?

I have a logistic regression model with 11 explanatory variables, 5 of which are dummy variables, when I use vif() function from library car in R, it gives me a VIF value for each of them. As far as I ...
user avatar
  • 163
0 votes
1 answer
62 views

High Variance inflation factor for all variables

As you can see there vif for all variables is too high...My interpretation is that all variables are highly correlated to each other. How can I further investigate on this? Or how can this be further ...
user avatar
  • 31
1 vote
0 answers
78 views

Variance of Linear Regression Coefficients in Terms of VIF

I am trying to understand how to rewrite the variance of regression coefficients in terms of the variance inflation factor. To be specific If we have a regression with design matrix X, I know that we ...
user avatar
  • 111
5 votes
1 answer
948 views

Why high correlation coefficient doesn't guarantee high VIF?

I know that there are similar questions about this topic, but my question is not really which metric should we depend on but why they are not equivalent? There is a saying in the book Basic ...
user avatar
  • 493
8 votes
2 answers
879 views

Isn't it futile to try to fix multicollinearity?

Most of the advice on how to deal with multicolinear predictors tells you to eliminate them before fitting your model, using some criterium like VIF (Variance Inflation Factor). If I understand it ...
user avatar
2 votes
1 answer
77 views

What to do in regression when an important continous covariate is correlated with the grouping covariate?

Hoping I can explain my problem clearly enough. My Masters thesis is looking at the impacts of elephant on woody vegetation in relation to distance from waterholes and vegetation type. Vegetation ...
user avatar
0 votes
0 answers
26 views

Is the arcsin trasformation useful to reduce multicollinearity?

I would like to use a GLM but, testing multicollinearity using the VIF parameter, I obtained higher correlation among independent variables (expressed as percentages). Is the arcsin trasformation ...
user avatar
0 votes
0 answers
1k views

Collinearity in factor dummies

I am running a linear regression on the house prices dataset from Kaggle and I was trying to see if I can find collinearity between some variables in order to improve my prediction. To this purpose ...
user avatar
  • 113
0 votes
1 answer
1k views

how to calculate VIF in logistic regression?

No book told me that... Using McFadden’s Pseudo-R2 ? OR do traditional linear regression to get VIF?
user avatar
  • 43
1 vote
0 answers
12 views

In what form should my features be in when running a model trained on features transformed through Weights of Evidence?

I have trained an anomaly detection model on features transformed/selected by means of the Weights of Evidence and Variance Inflation Factor approaches. My question is how should I go about running ...
user avatar