Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

0
votes
0answers
17 views

VIF interactions

I would like to check for multicollinearity in logistic regression analysis. Independent variables are categorical (always binary) and continuous. Sample has limitted size (N=176, 36 events), so I can ...
0
votes
0answers
7 views

Univariate analysis, Multicollinearity

I am developing multivariate binary logistic regression model in order to find out risk factors for death (outcome). Should I use univariate logistic regression in order to "reduce" predictor´s list ...
0
votes
2answers
26 views

log log model: multicollinearity and interpretation

I would like some advice on a small multiple regression model. The model is a log - log one, with log(investments) as the dependent variable. My issue is that I would like to introduce log(GDP per ...
0
votes
1answer
19 views

Collinearity testing between predictors

I would like to test a collinearity between possible "predictors (risk factors)" for binary outcome (death). Possible "predictors" are categorical (always binary) and continuous... For two ...
2
votes
1answer
52 views

Lasso will not remove correlated variables

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
0
votes
1answer
15 views

What is the difference/relation between variables that are multicollinear, confounding, interacting

What is the exact difference between two variables that are multicollinear and two variables that are interacting and two variables that are confounding? Are they multiple meanings for the same ...
0
votes
0answers
23 views

Colinearity and eigenvalue

I'm doing a binary regression with two predictors, one continuous and one binary. To check for collinearity I looked at the correlation .444, at the VIF 1.13 and 1.18 and the % of eigenvalue share. ...
1
vote
0answers
8 views

Shapley Value Regression for prediction

I've been successful in using the relaimpo-Package for R in SPSS through STATS_RELIMP to calculate the Importances of different predictors (in cases of multicollinearity). What im wondering now is how ...
0
votes
0answers
10 views

Does the variance inflation factor make sense for regularized regression?

I have a logistic regression model fit using L1 regularization. There are two variables that entered the model that have a correlation of over 0.90. The VIF for these variables are each about 60 which ...
6
votes
3answers
264 views

Strange outcomes in binary logistic regression in SPSS

I did a binary logistic regression with SPSS 23 and I found some strange outcomes. This is for NOACprev until No_Prev_treatment, the last 6 variables. First of all they have very high outcomes for B, ...
0
votes
0answers
21 views

Statistic test for correlation among multiple variables || statistical test for confirming sales pattern among multiple items

While thinking about sales of a company, I came across a question. I have data like below: Day1 day2 ......day30. item1 10. 20. ....... 30. ...
0
votes
1answer
42 views

Multicollinearity - continuous and dummy variables

I know that one of the assumptions of Gauss-Markov is no perfect multicollinearity. If I want to run a model that estimates the effect of gambling on wages, would this model be appropriate: Wage = ...
0
votes
3answers
52 views

Does practical insignificance mean no relationship?

I have two problems: 1) I have a regression coefficient that is very significant (large dataset), but has low practical significance. Can I say there is no relationship? And I mean really tiny, like ...
0
votes
0answers
16 views

How to control serially correlated independent variables?

I'm interested in studying the impact of one variable (e.g., R&D expense at year T) on future firm performance (e.g., Sales in year T+5), I know it's incorrect to specify the following model: ...
0
votes
0answers
8 views

Gross capital and consumption expenditure

I am doing a multivariate regression with gross capital formation and final consumption expenditure as the dependent variables. This is more in the realm of economics, but will the nature of the two ...
1
vote
1answer
20 views

Can feature with high positive correlation have opposite weights sign? [duplicate]

I have two features for a binary classification problem which are highly positively correlated. (0.79) But when I build a logistic regression classification then I see their weights are opposite in ...
2
votes
0answers
18 views

Should the intercept be included when you check the condition index?

Many sources state that a condition index >30 constitutes a multicollinearity problem. When I've tried to implement this check in practice, I've realized that the condition index (and VIFs) change ...
1
vote
0answers
23 views

Perfect collinearity between one level of two categorical variables

I am setting up a logistic regression model with mostly categorical independent variables (answers to survey questions). Some of the variables have levels like "High-Medium-Low-NotApplicable". The ...
7
votes
2answers
79 views

Why aren't we simply using $R_j^2$ instead the VIF?

After all we calculate the VIF by $1/(1-R_j^2)$. A VIF of $5$ corresponds to an $R_J^2$ of $0.8$. To me, the information given by $R_j^2$ just becomes more obscure when I apply the VIF formula. Why ...
4
votes
1answer
346 views

Why are VIFs below ten not still considered very worrying?

I've been trying to read up on multicollinearity, and I think I have a decent grasp of it, and of what VIF tells me. But there is one aspect of the advice that seems quite universal, but makes me ...
0
votes
0answers
24 views

Understanding multicollinearity and bias in coefficient estimates

In trying to better understand the effects of multicollinearity within the context of logistic regression, I have come across the following quote from Paul Allison's textbook: Although ...
2
votes
1answer
29 views

Is it OK to use an original variable & another variable constructed from it in a regression model if there is no multicollinearity?

I'm doing binary logistic regression. I want to predict the chance of being in an advanced class. There no multicollinearity among my variables. I have three predictors: If you passed the test or ...
0
votes
0answers
22 views

Potential multicollinearity problem between a dummy and the constant term

In an OLS regression, I need to use a variable that is equal to 0 for all observations (1636) except 3 of them. I am afraid that this may generate a multicollinearity problem with the constant of the ...
0
votes
0answers
14 views

How does a non-random sample limit Logistic Regression

I have been attempting to run a binary logistic regression and have come into some difficulties with it. The outcomes I have got are OK for odds ratio, but the space between the lower and upper ...
1
vote
1answer
51 views

Collinearity in Classification Model for Churn Prediction

I'm working on evaluating various classification algorithms to help predict customer churn (or at least ID interesting features to use in later strategy). The goal is to identify accounts who are at ...
0
votes
2answers
70 views

Should multicollinearity problem be looked into while doing cointegration?

Multicollinearity and Cointegration is not the same thing however, if the series actually move together in the long-run i.e. are cointegrated wont they also be collinear making e.g. Autoregressive ...
0
votes
0answers
26 views

Issues of collinearity in Granger Causality testing

If I have a regression where a lagged explanatory variable is regressed on a dependent variable, and I add to this regression lead explanatory variables (i.e. future lagged explanatory variables) in ...
1
vote
0answers
35 views

How to interpret differences in VIF and condition number?

In my present data, the Variance Inflation Factors suggest lack of substential multicolliniearity (<1,7). However, the condition number of 28 is almost at the critical value of 30. How do I ...
2
votes
0answers
19 views

Do I have multicollinearity? [duplicate]

I am examining the impact of 7 IVs on one DV using regression anaysis. Some of the IVs are significantly correlated with each other, which is consistent with theory. While the single OLS regression ...
0
votes
1answer
77 views

How to detect multicollinearity in a logistic regression where all the dependent variables are categorical and binary?

I'm doing a multivariate logistic regression where all my independent variables are categorical and binary. I have transformed all my categorical variables into dummies in order to have reference ...
0
votes
0answers
12 views

Is it possible to check multilevel growth curve models for multicollinearity?

I'm modelling a growth curve model, based on five-time points for n=243 individuals. Time is treated flexible rather than occasion specific. Next to the dependent, continuous variable, I want to ...
0
votes
0answers
40 views

how to handle very correlated variables

I regress five variables x1-x5 on y using OLS, VECM, ARDL (Im trying to learn all of them). However x1 and x2 are highly correlated and due to multicollinierity the slope coefficients for x1 change ...
0
votes
1answer
34 views

Feature selection based on mean, standard deviation and mean absolute deviation

Suppose we have a large dataset (~ 60000 entries, 58 variables, 4 class labels). For each variable mean, standard deviation and mean absolute deviation are calculated - separately for every class ...
1
vote
0answers
55 views

Can removing predictors increase predictive power in large sample cases?

I am observing an increase in the predictive power of my logistic regression when I remove certain predictors. It might be a bug in my code, so I'm wondering: is this statistically possible? If so, ...
0
votes
1answer
31 views

Is it correct to use sampling weights for reducing multicollinearity in Probit?

I tried to estimate a probit model: $$y=b_1x_1+b_2x_2+b_3x_3,$$ where all variables are string and $x_1$ and $x_2$ are correlated. $x_1$ means GINI index and $x_2 = \text{self-employment rate}\cdot ...
0
votes
0answers
32 views

Why do t statistics decrease (standard error for cofficient increases) when multicollinearity exists?

Can anybody show how the t statistic decreases when multicollinearity exists? It is easy to prove using the F test, but I don't know how to prove it using the t test.
1
vote
1answer
34 views

Different coefficient values from multiple versus bivariate regression

I wonder how to generate such data, so that in single variable regression feature coefficient would be positive, and in multiple regression would be negative. So I read several related questions on ...
14
votes
1answer
986 views

Why does this regression NOT fail due to perfect multicollinearity, although one variable is a linear combination of others?

Today, I was playing around with a small dataset and performed a simple OLS regression which I expected to fail due to perfect multicollinearity. However, it didn't. This implies that my ...
0
votes
0answers
36 views

Vif and stepwise regression

I used the VIF to detect multicollinearity, I want to use forward selection and backward elimination procedures. My question is: Do I have to use all the variables in my dataset in the procedures or ...
3
votes
0answers
17 views

reconciling Linearity and Multicollinearity assumptions in ANCOVA

For ANCOVA, many textbooks and other resources require, among other assumptions, that covariates be linearly related to the dependent variable, which makes sense. However, many of the same sources ...
0
votes
0answers
35 views

Issue with linear mixed effects model and interaction : any alternate method?

I have a variable (Y) which I'd like to know if it can be explained by linear regression with two other variables (A and B) or the interaction of A and B. Let's say, to simplify, than I have two ...
1
vote
1answer
55 views

Is it appropriate to test for collinearity in a mixed model using VIF?

My study is examining predictors of skin lesions in pigs. I am looking at the effect of predictor variables (including weight at 4 weeks, 9 weeks and 20 weeks) and I have carried out a mixed model ...
1
vote
0answers
44 views
1
vote
0answers
34 views

Is the intercept estimation affected by multicollinearity?

Suppose I am running a regression $$x_t = \alpha + b_1y_{1t} + \dots + b_m y_{mt} + \varepsilon_t$$ where the $y_{i}$ are potentially linearly correlated (Some have an IVF bigger than 4; generally ...
1
vote
0answers
41 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
-1
votes
1answer
83 views

Adding new variables makes regression coefficients individually insignificant [duplicate]

I have a multiple regression where all my coefficients are significant. When I add new variables my initial variables become insignificant. Furthermore, my new variables (that in a simple regression ...
2
votes
1answer
54 views

Regression — multicolinearity and VIFs

I understand that variance inflation factors can be used to detect multicollinearity. What is the intuition behind the VIF formulation? What aspect of this formula shows it detects multicollinearity ...
0
votes
0answers
13 views

Nonlinear least squares for multicollinear data - regularization etc?

Is there an existing technique that can be used to fit NLLS on multicollinear data? Such as ridge regression for NLLS, or any other technique used to solve the same problem? My model takes the form ...
0
votes
1answer
53 views

Does multicollinearity affect performance of a classifier?

I know that wikipedia references are sometimes frowned upon here, but this one has me puzzled: Wikipedia - Multicollinearity I know what multicollinearity is, and ...
1
vote
0answers
84 views

Alternatives to Box-Tidwell transformation for ridge regression?

I would like to fit the following model by ridge regression (the xs correlate strongly with one another) $y = \beta_1 {x_1}^{\lambda_1} + \beta_2 {x_2}^{\lambda_2} + \beta_3 {x_3}^{\lambda_3} + ...