Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
4
votes
1answer
1k views

VIF Drops Significantly When I Delete Some Dummy Variables

Is my model valid even with the high VIF? Does it matter which dummy variable I drop as the reference point? I have a a category variable (Fruit) that I converted ...
4
votes
2answers
2k views

Why there is a dependence between the factors on the same column?

I found that the lm model I have trained has some NAs in the coeficents, and it is in the factor columns. So I have searched for aliases an I have found that some factor values depend on the other ...
4
votes
1answer
157 views

Principal Component Regression with an additional factor

I am looking to tease out the significance and contribution of a particular variable to 2 different continuous responses. I have 7 continuous variables I know to be influential on the two responses (...
4
votes
2answers
1k views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
4
votes
2answers
3k views

I have my data set, now what?

I have a basic understanding of basic statistics, but I believe I've gotten myself out of my depth. I have a data set with a dependent variable (time span) and three quantitative independent ...
4
votes
3answers
889 views

VIF understanding - does only >4 variables are multi-collinear and others are not?

I am trying to understand if there will be multicollinearity between few variables or not. I took a sample data and tried to see the Variance Influencing Factor results - in general vif > 4 indicates ...
4
votes
1answer
219 views

Multicollinearity in linear regression vs interpretability in new data

I am working in a project using linear regression with a data set that has a lot of multicollinearity. From what I could understand from my research about the topic, I can separate the issues caused ...
4
votes
1answer
1k views

Multilinear regression with multicollinearity: residual regression

I am trying to build a multilinear regression with predictor variables that likely are correlated. I understand that this is a problem, due to overlapping explanations of data. I think I have a method ...
4
votes
2answers
688 views

Multicollinearity Using VIF and Condition Indeces

I am testing my dataset for multicollinearity using VIF and condition indices(CI).My dataset is cross-sectional macroeconomics data. I have 6 independent variables ($x_1$,$x_2$,$x_3$,$x_4$,$x_5$,$x_6$)...
4
votes
4answers
1k views

Multicollinearity and categorical predictor with three levels

If I have a continuous Dependent Variable and two Independent Variables, where one is categorical with three levels and the other is continuous, what assumptions do I need to check for multiple ...
4
votes
1answer
176 views

Which data for feature selection to get unbiased result?

I have a 70 / 30 ratio for train / test data. I have a relatively small feature set (6 features), however, I still want to do feature selection to get rid of any redundant features (I'm guessing 1 of ...
4
votes
1answer
844 views

backwards stepwise regression, collinearity and regression to the mean

My research paper was recently rejected and some of the feedback I received was in relation to the statistical tests done/not done. I would like help in clarifying what I could do differently as the ...
4
votes
1answer
457 views

higher order polynomial fits do not match training data

I am fitting a high order polynomial fit (order 15+) to some simulated training data. I know that features become collinear as i increase the order of polynomial but i do not undersand why my fits are ...
4
votes
2answers
2k views

How do I use a correlation matrix in Big Data?

I'm fairly new to Big Data and have been reading the book 'Applied Predictive Modeling' by Max Kuhn, Kjell Johnson. I'm trying to understand how to use the correlation matrix in the context of big ...
4
votes
1answer
1k views

Collinearity testing between predictors

I would like to test a collinearity between possible "predictors (risk factors)" for binary outcome (death). Possible "predictors" are categorical (always binary) and continuous... For two ...
4
votes
2answers
4k views

Should multicollinearity problem be looked into while doing cointegration?

Multicollinearity and cointegration is not the same thing; however, if the series actually move together in the long-run i.e. are cointegrated, won't they also be collinear, making e.g. autoregressive ...
4
votes
1answer
974 views

Non negative least squares with minimal colinearity

I am trying to fit a dataset using the standard NNLS (non-negative least squares) approach. Formally: $\min_x ||Ax-b||^2_2$ s.t. $x\ge0$ This is a quadratic program and can be solved optimally. The ...
4
votes
0answers
107 views

Cox interactions and co-linearity

I am interested in developing a model to predict survival based on a few predictors: age, sex, and two lab values, albumin and globulin. I have approximately 14,000 deaths in the data. I initially ...
4
votes
0answers
2k views

Multicollinearity: How to convert GVIF^(1/(2df)) values to VIF

I am using the GVIF^(1/(2df)) method in my analyses to check for multicollinearity of my (mainly) categorical variables. However, I am struggling with the cut-off values. For the 'regular' VIF several ...
4
votes
0answers
383 views

When including a linear interaction between two continuous predictors, should one generally also include quadratic predictors?

Suppose I am fitting a linear model, and I have two continuous predictors x1 and x2. I think that they might interact, so I add ...
4
votes
0answers
238 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
4
votes
0answers
664 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
4
votes
0answers
8k views

How to diagnose multicollinearity using the output of vif function in R?

I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model. When I run vif() on my final model, I get <...
4
votes
0answers
6k views

Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
3
votes
2answers
3k views

How seriously should I consider the effects of multicollinearity in my regression model?

I have a model y ~ x + z and the correlation between x and z is 0.2. This is only weakly positive. So, how seriously should I consider the effects of ...
3
votes
3answers
2k views

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? [duplicate]

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? My data is large enough (i.e. I have several regression models where each of the data points for them ...
3
votes
2answers
3k views

Multicollinearity when adding a confounding variable

When you run a regression on ice cream sales with predictor shark attacks, you find a significant coefficient. But that is because there is a confounding variable temperature. But how do you correct ...
3
votes
1answer
5k views

What is collinearity and how does it differ from multicollinearity?

I was reading this when I came across the term collinearity. I tried looking up what it is but top results are related to multicollinearity. I could find here about multicollinearity ...
3
votes
1answer
306 views

How to approximately solve system of linear equations that has no solutions

When I prepare data for Bayesian network meta analysis using model published in Woods B.S, Hawkins N et al - Network meta-analysis on the log-hazard scale, combining count and hazard ratio statistics ...
3
votes
2answers
101 views

Physically implausible results in linear regression with colinearities

While developing a model (a Poisson regression, but this is not the topic of this post), I stumbled upon a physically implausible relationship between some variables. I have ground temperature data ...
3
votes
3answers
97 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
3
votes
1answer
1k views

Multicollinearity in simple linear regression (not multiple)?

I am doing a simple linear regression analysis with 1 independent variable. I am checking data against assumptions. As I am checking against Tolerance and VIF level, I get the their values equal to 1 (...
3
votes
5answers
7k views

Test independence between quantitative and categorical predictors for logistic regression

I have 2 categorical variables with 8 unordered categories and multiple numerical variables and I want to train a logistic regression model. I want to test the independence between all my predictor ...
3
votes
3answers
2k views

Overall prediction using correlated variables

I have a large set of data and a couple of regressors that seem to be somewhat to highly correlated. I will include these in a GLM and am primarily interested in the predictive ability of the model ...
3
votes
1answer
258 views

Why is only linearity between predictors a problem?

in these days I was thinking about the collinearity or multicollinearity problem in a linear regression model. Let's suppose we have a model like this: $Y_i=\beta_0+\beta_1X_1+\beta_2X_2+\beta_3X_3+\...
3
votes
2answers
1k views

Correlation and collinearity in regression

I did a correlation analysis for my variables. All of them are associated (the coefficient is above 0). However, there is no collinearity problem in my regression analysis. I do not know how to ...
3
votes
4answers
1k views

Interpretation of regression coefficients in the presence of modest correlations

I have a multiple regression model where I have nearly 20 independent variables. These variables are modestly correlated with each other (e.g., the maximum VIF is around 4 with most of them in the 2s)....
3
votes
2answers
5k views

Is a log transformation of predictors a suitable way of dealing with multicollinearity in multiple regression?

Suppose two independent variables in the linear regression initially have very high correlation of 0.95. This introduces severe multicollinearity into the model (as indicated by very high variance ...
3
votes
1answer
771 views

Mean centering interaction terms

I want to mean center my interaction terms in a regression model (i.e., make the mean zero for each variable). I understand that I am supposed to mean center my variables first and then multiply them ...
3
votes
3answers
663 views

Does multicollinearity cause type I errors?

Citing Wikipedia on multicollinearity, One of the features of multicollinearity is that the standard errors of the affected coefficients tend to be large. In that case, the test of the hypothesis ...
3
votes
1answer
914 views

Infer causality with high collinearity

I recently started to ask myself how to measure the impact of education on indexes like GDP: what is the outcome of mathematics or computer science on GDP, at the country level for instance. In this ...
3
votes
2answers
3k views

Comparing logistic regression models with same number of parameters

What would be an appropriate way to compare two logistic regression models with the same number of parameters (i.e., model 1 is not nested in model 2)? In my case, I cannot compare the models to a ...
3
votes
1answer
767 views

Two negative beta's in a curvilinear regression when mean centered or using standardized values

The problem I encounter is the following: Imagine a (perfect) inverted U-shaped relation between an independent variable and a dependent variable. When you look at the curve estimation there is ...
3
votes
1answer
634 views

What can go wrong if I include two categorical variables and intercept in linear regression?

What can go wrong if I include two categorical variables and intercept in linear regression? With: y~x1+x2 Both x1 and ...
3
votes
2answers
364 views

Estimating the effect size in polynomial regression

In polynomial regression, estimating the effect of an independent variable on the outcome seems to be quite tricky (to me). For example: I want to compare the influence of $x$ on $z$ with the ...
3
votes
2answers
1k views

Failing to reject the null Hypothesis in the face of multicollinearity

I don't understand very well the statement that in the face of multi-collinearity we may fail to reject the null hypothesis. The Null Hypothesis is that the coefficient of a variable is zero. This ...
3
votes
1answer
942 views

How to deal with interaction term's VIF score

I have a linear regression model that has no multicolinearity problem with low VIF scores. However, when I include the interaction term, this interaction term and its components get very high VIF ...
3
votes
1answer
5k views

How can we calculate the variance inflation factor for a categorical predictor variable when examining multicollinearity in a linear regression model?

For example, if we have the linear regression model: $$E(y) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 $$ where $ x_1 =\begin{cases} 1 & \mbox{if level 2} \\ 0 & \mbox{otherwise} \...
3
votes
1answer
2k views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be non-...
3
votes
1answer
926 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...

1 2 3
4
5
18