Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

2
votes
2answers
42 views

Why do both the VIF and tolerance statistics exist, when the latter is just the reciprocal of the former?

Is taking the reciprocal helpful in some way, or is it just a matter of historical accident that there came to be two terms to describe the same thing? (From the Wikipedia page.)
1
vote
1answer
34 views

Multicollinearity if independent variables sum to one?

I am intending to explain the variation in dependent variable Y through a number of ratios that, combined, sum to 1 in each period t Hence, $$ Y_t = \alpha + ...
4
votes
0answers
35 views

Variance inflation factor for generalized additive models

In the usual VIF calculation for a linear regression, each independent/explanatory variable $X_j$ is treated as the dependent variable in an ordinary least squares regression. i.e. $$ X_j = \beta_0 + ...
3
votes
0answers
20 views

When including a linear interaction between two continuous predictors, should one generally also include quadratic predictors?

Suppose I am fitting a linear model, and I have two continuous predictors x1 and x2. I think that they might interact, so I add ...
1
vote
1answer
59 views

Dealing with multicollinearity in logistic regresson model

I have 5 quantitative predictor variables in my logit model, and when I use the cor function in R on those 5 variables, I see that $x_{1}$ and $x_{2}$ have ...
1
vote
0answers
37 views

Infer causality with high collinearity

I recently started to ask myself how to measure the impact of education on indexes like GDP: what is the outcome of mathematics or computer science on GDP, at the country level for instance. In this ...
0
votes
0answers
19 views

handling multicollinearity by backwards regression and omitted variable bias

Suppose I try to estimate a production function as follows: $logY=b_0+b1*logX_1+b_2*logX_2+b_{11}*(logX_1)^2+b_{22}*(logX_2)^2+b_{12}*(logX_1)*(logX_2)+u$, where $Y$ is the output, $X_1$, $X_2$ are ...
4
votes
0answers
25 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
1
vote
0answers
25 views

How to deal with imperfect multicollinearity

I am estimating a model for an assignment, and found out that there is a 0.9 correlation between two of the independent variables. So If I am not wrong I should omit one variable and redo the ...
4
votes
2answers
141 views

I have my data set, now what?

I have a basic understanding of basic statistics, but I believe I've gotten myself out of my depth. I have a data set with a dependent variable (time span) and three quantitative independent ...
0
votes
1answer
60 views

Collinearity in time dummies, fixed effect regression

I am running a fixed effect panel regression with 81 groups x 20 periods, so approx 1620 (unbalanced) observations. I use the following to create dummies: ...
1
vote
0answers
49 views

Dealing with multicollinearity by smoothing coefficients?

I would like to perform a linear regression. However, the predictor variables are families of variables indexed by time. Let's say the regression problem is: target ~ ...
4
votes
3answers
207 views

Regression for really small data with high degree of multicollinearity and outliers

I'm working on a promotional response analysis. I have a really small real world dataset with 25 observations and 15 variables. The variables have a high degree of multicollinearity and some have ...
1
vote
0answers
35 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale IVs and 2 categorical IVs and 1 scale DV. I am testing the assumptions of regression and I a not sure how to test multicollinearity since I ...
0
votes
1answer
119 views

Confused about multicollinearity, variable selection and interaction terms

I have run a few tests/methods on my data and am getting contradictory results. I have a linear model saying: reg1 = lm(weight = height + age + gender (categorical) + several other variables). If I ...
1
vote
1answer
51 views

Testing for multicollinearity in logistic regression

So far I have checked the tolerance value, VIF and condition indexes. But checking the variance of the regression coefficients I have to wonder: how little variance of the regression coefficient ...
0
votes
0answers
26 views

Utilizing PCA when rotated factor loadings are difficult to relate to real world variables?

In my ongoing attempts to make sense of a highly correlated dataset without just throwing out most of the variables i am at the point of attempting to interpret PCA output. I hope i am right in the ...
5
votes
3answers
110 views

What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
0
votes
0answers
20 views

PCA, non positive definitive & potential cures?

I am trying to use PCA to remove the correlation in my dataset My correlation matrix is being reported as non positive definitive As I understand it a NPD report means that my transposed matrix does ...
0
votes
0answers
15 views

Correlated varaibles and bootstrap forest possible problem?

I have been using this method to reduce the number of variables for a model but i have just had a bad thought. Due to the nature of bootstrap is it possible that by randomly using variables you are ...
1
vote
0answers
56 views

Is it possible to center orthogonal polynomials in multiple regression

I have a regression model that looks like the one below. $$ Y = \beta_0 + \beta_1T + \beta_2T^2 + \beta_3T^3 + \beta_4D + \beta_5D*T + \beta_6D*T^2 + \beta_7D*T^3 $$ Where ...
2
votes
0answers
208 views

How to test and avoid multicollinearity in mixed linear model?

I am currently running some mixed effect linear models. I am using the package "lme4" in R. My models take the form: ...
1
vote
0answers
46 views

Multicollinearity of Socioeconomic index vs. wealth variables separately?

I am running Cox regressions in a large dataset to determine effects of individual and area characteristics on mortality outcomes. (My model is not a multilevel model, as the initial significant ...
0
votes
1answer
61 views

Collinearity among explanatory variables

My correlation matrix is not showing very high correlations (less than 0.5), but still, when I am regressing panel fixed effects models, all my variables are omitted, and it shows it is due to ...
1
vote
2answers
89 views

Multicollinearity when adding a confounding variable

When you run a regression on ice cream sales with predictor shark attacks, you find a significant coefficient. But that is because there is a confounding variable temperature. But how do you correct ...
0
votes
0answers
35 views

Confirming multicollinearity with small number predictors (after initial model construction)

I am working on a dataset with with 100 or so X variables and the effects on a single Y. As with previous threads this contains a lot of fairly significant correlations. I considered using PCA but ...
0
votes
1answer
41 views

Fitting same classes of many many collinear variables

I am trying to build a linear model of the simple form, y = a + b1*x1 + b2*x2 + b3*x3 + c1*z1 + c2*z2 + c3*z3.... The characteristics of the model are 1) Each of the independent variable belongs to ...
2
votes
0answers
53 views

Question about variance inflation factors

I'm considering the regression model $y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \varepsilon_i$ where the $\varepsilon_i$ are iid and $\mathcal N(0,\sigma^2)$ A study question asks to show the ...
0
votes
0answers
32 views

multicollinearity test

I use several control variables and use independent variable $A$ for model 1, use independent variable $B=1-A$ for model 2, use independent variable $C=A-E-F$ for model 3, use independent variable ...
1
vote
1answer
172 views

Handling redundant factor variable levels for linear regressions in R

Say I have two factor variables, X and Y, each with 3 levels. However, X==3 if and only if Y==3, while such a connection doesn't hold for X,Y==1,2. In this case, while X and Y are not redundant, my ...
1
vote
0answers
25 views

Estimability and less than full rank model matrices

So I have some data... ...
0
votes
0answers
110 views

Using square of explanatory variables when the correlation is high

I have explanatory variable age in the model (along with others) and there are several theoretical justification for using square of age variable (along with age) in the model,e,g,here (if we have a ...
4
votes
2answers
191 views

Logistic Regression and how to judge model fit and parameter influence

I want to show statistical significant impact of about 15 different independent variable on a binary dependent variable (I am not a statistician). Some of my independent variables are term counts in ...
1
vote
0answers
55 views

Ridge regression not appropriate for collinearity caused by mathematical constraints on the data

In this paper: Use of the Bootstrap and Cross-Validation in Ridge Regression Author(s): Nancy Jo Delaney and Sangit Chatterjee Source: Journal of Business & Economic Statistics, Vol. 4, No. 2 ...
2
votes
1answer
199 views

Effect of Wald-test and collinearity on Logistic Regression model selection

A researcher is interested in how variables, such as GRE (continuous), GPA (continuous) and rank of the undergraduate institution (categorical), affect admission into graduate school. The response ...
3
votes
0answers
59 views

Should I de-mean a predictor variable before a dummy interaction

Suppose I have the following time-series linear model where $\beta$ is misspecified: $Y(t+1) = \alpha + \beta X(t) + \sum_{i=1}^{10000}\gamma_i Z_i(T) + \varepsilon$ where all parameters are in ...
3
votes
2answers
166 views

Intuition for consequences of multicollinearity

So we have a regression equation with one explained variable and 10 explanatory variables. What I have read so far: Multicollinearity doesnt affect the regression of the model as a whole. But if we ...
0
votes
0answers
16 views

Matroid techniques for identifying multicollinearity

Reading Belsey (Conditioning Diagnostics 1991) he mentions on page 35 what was then a new technique of using the flats of the Matroid of dependencies in X to identify multicollinearity see Greene ...
2
votes
1answer
177 views

Diagnosing collinearity in a Cox proportional hazards model

I am building a Cox Proportional Hazards Model to predict the survival outcome of seabird faced with predation pressure. I have 6 factor variables with two or three levels each that I have predicted ...
1
vote
2answers
119 views

Interpretation of regression coefficients in the presence of modest correlations

I have a multiple regression model where I have nearly 20 independent variables. These variables are modestly correlated with each other (e.g., the maximum VIF is around 4 with most of them in the ...
3
votes
1answer
115 views

How to deal with collinearity in lme with categorical IV with > 2 levels

I'm analysing data from our experiment. We had participants in 4 groups, each participant was measured 4 times. We measured cortisol in saliva, so it leads us to the linear mixed models, because the ...
1
vote
1answer
295 views

Binary Logistic Regression Multicollinearity Tests

I like Peter Flom's answer to an earlier question about multicollinearity in logistic regression, but David Garson's Logistic Binomial Regression states that there is no valid test for ...
2
votes
2answers
133 views

Variable selection with groups of predictors that are highly correlated

What variable selection approach should I consider if I have thousands of predictors with clusters that are extremely correlated? For example I might have a predictor set $X:= ...
1
vote
1answer
37 views

Case-control study with data collected in batches

I have (matched) case-control data. The data is collected in batches in such a way that the batch determines some quality of the data (there is a 'batch' effect). Also, the cases and controls were ...
0
votes
2answers
146 views

Comparing logistic regression models with same number of parameters

What would be an appropriate way to compare two logistic regression models with the same number of parameters (i.e., model 1 is not nested in model 2)? In my case, I cannot compare the models to a ...
2
votes
0answers
109 views

Why is there not always a collinearity problem when introducing interaction terms to a MLR or logistic model?

Collinearity is described as occurring when an independent variable is a linear function of another variable(s). Such that the variable in question can be consistently predicted given the other ...
1
vote
1answer
82 views

Unstable coefficient in regression without high correlation between variables

I am estimating a linear regression: $Y=f(X_1,X_2,X_3,X_4,X_5)$. My test shows that when the equation includes $X_4$ and $X_5$ only, $X_4$ is not statistically significant ($t$-value=1.26). However, ...
8
votes
1answer
183 views

Multicollinearity and Splines :: Is there a problem?

When using natural (i.e. restricted) cubic splines, the basis functions created are highly collinear, and when used in a regression seem to produce very high VIF (variance inflation factor) ...
9
votes
1answer
1k views

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...