Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
2
votes
1answer
650 views

How to compare parameters between different types of generalized linear model?

I am doing some work on the effects of collinearity on different types of model (OLS, binomial logistic, ordinal logistic, multinomial logistic and maybe others). I have found the perturb package in R,...
9
votes
1answer
3k views

Multiple regression with missing predictor variable

Suppose we are given a set of data of the form $(y,x_{1},x_{2},\cdots, x_{n})$ and $(y,x_{1},x_{2},\cdots, x_{n-1})$. We are given the task of predicting $y$ based on values of $x$. We estimate two ...
0
votes
1answer
340 views

Significant predictor loses significance when second non-significant predictor is entered (log regression) [duplicate]

Possible Duplicate: How can adding a 2nd IV make the 1st IV significant? I am looking at the prediction of relapse by gender. In a hierarchical log regression, with gender entered in first block,...
1
vote
1answer
673 views

Multinomial logistic regression?

How can I look for collinearity diagnostics, particularly condition indexes and proportions of variance in Multinomial logistic regression?
13
votes
2answers
6k views

What are chunk tests?

In answer to a question on model selection in the presence of multicollinearity, Frank Harrell suggested: Put all variables in the model but do not test for the effect of one variable adjusted ...
5
votes
1answer
2k views

How to deal with multicollinearity issue when analyzing survey results?

I recently conducted my first study looking at parental education as a determinant of risk. I conducted a survey which measured risk in several contexts and also recorded information about the ...
29
votes
3answers
40k views

How to deal with multicollinearity when performing variable selection?

I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
2
votes
1answer
564 views

How to choose the best of two highly correlated predictors in cox proportional hazards regression

I have built a good model of time-to-stroke under cox ph assumptions using a predictor of stroke risk (Framingham risk score). It incorporates a score according to Age, Gender, controlled / ...
6
votes
1answer
767 views

Abusing Linear Models under Multicollinearity: Simulation for 'realistic' movement of predictors

I have a reasonable understanding of why multicollinearity is a problem is regression models, along the lines of this excellent post. To summarise my understanding, for a regression model of $y = \...
13
votes
6answers
9k views

Multicollinearity when individual regressions are significant, but VIFs are low

I have 6 variables ($x_{1}...x_{6}$) that I am using to predict $y$. When performing my data analysis, I first tried a multiple linear regression. From this, only two variables were significant. ...
2
votes
1answer
2k views

Residualize a binary variable to remedy multicollinearity?

Imagine a regression model where there is a continuous-valued response variable and three continuous-valued explanatory variables. For concreteness, imagine that we are interested in the effects of "...
2
votes
2answers
782 views

Model averaging in prediction — “Wisdom of the Crowd”

Suppose I'm trying to predict $Y$ (a real number) and I have $n$ experts with guesses $Y_1,...Y_n$. Each prediction is a reasonable guess as to the value of Y in itself (hence the name "expert"), but ...
4
votes
3answers
1k views

Does the p-value in the incremental F-test determine how many trials I expect to get correct?

I've implemented an incremental F-test program that evaluates the fit of an unrestricted model $M_{UR}$ against the restricted model $M_R$ using the F statistic $\frac{SSE_{R} - SSE_{UR}}{SSE_{UR}}\...
5
votes
2answers
3k views

Understanding condition index used for finding multicollinearity

In linear models, in my book, the condition index is defined as $\sqrt{\lambda_{max} \over \lambda_{min}}$ where $\lambda_{max}$ is the maximum eigenvalue of $ZZ^*$, i.e., the correlation matrix of ...
1
vote
1answer
516 views

Assessing multicollinearity of factors

I have a multifactor model (with 7 factors currently) and 754018 observations. In order to check for multicollinearity issues as the model grows I wrote an R script to compute a correlation matrix ...
2
votes
0answers
108 views

multicollinearity question, X and X change score

Consider survey data from surgeries. $Y$ represents observed surgical quality and is measured post-surgery; $X$ represents perceived surgical difficulty level and is measured pre and post surgery. ...
12
votes
2answers
20k views

Collinearity between categorical variables

There's a lot about collinearity with respect to continuous predictors but not so much that I can find on categorical predictors. I have data of this type illustrated below. The first factor is a ...
14
votes
5answers
7k views

Does standardising independent variables reduce collinearity?

I've come across a very good text on Bayes/MCMC. IT suggests that a standardisation of your independent variables will make an MCMC (Metropolis) algorithm more efficient, but also that it may reduce (...
2
votes
1answer
5k views

How to interpret VIF and Condition Index for the purpose of assessing reliability of formative measures?

I am testing the reliability of my formative measurement model and I am using Variance Inflation Factor (VIF) and Condition Index (CI) (see this earlier question asking whether to and how to do this). ...
81
votes
0answers
64k views

How can a regression be significant yet all predictors be non-significant? [duplicate]

My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant. All the regression assumptions are met. No multicollinearity ...
13
votes
2answers
4k views

Dealing with multicollinearity

I have learnt that using vif() method of car package, we can compute the degree of multicollinearity of inputs in a model. From ...
11
votes
1answer
460 views

Reference for the sum and difference of highly correlated variables being almost uncorrelated

In a paper I've written I model the random variables $X+Y$ and $X-Y$ rather than $X$ and $Y$ to effectively remove the problems that arise when $X$ and $Y$ are highly correlated and have equal ...
1
vote
0answers
2k views

How to deal with negative coefficients in logistic regression

I have 5 single factor regression models. Each has intuitive (positive) coefficients which is good. If I run multiple logistic regression one of the coefficients turns negative. If I run stepwise ...
29
votes
2answers
17k views

Is PCA unstable under multicollinearity?

I know that in a regression situation, if you have a set of highly correlated variables this is usually "bad" because of the instability in the estimated coefficients (variance goes toward infinity as ...
8
votes
2answers
1k views

Assessing multicollinearity of dichotomous predictor variables

I'm working on a project where we observe behaviour on a task (eg. response time) and model this behaviour as a function of several experimentally manipulated variables as well as several observed ...
5
votes
3answers
2k views

Can you use heteroskedastic time series variables within a regression model?

We are working on a multivariate linear regression model. Our objective is to forecast the quarterly % growth in mortgage loans outstanding. The independent variables are: 1) Dow Jones level. 2) % ...
21
votes
1answer
14k views

Logistic Regression - Multicollinearity Concerns/Pitfalls

In Logistic Regression, is there a need to be as concerned about multicollinearity as you would be in straight up OLS regression? For example, with a logistic regression, where multicollinearity ...
16
votes
2answers
30k views

VIF, condition Index and eigenvalues

I am currently assessing multicollinearity in my datasets. What threshold values of VIF and condition index below/above suggest a problem? VIF: I have heard that VIF $\geq 10$ is a problem. After ...
24
votes
6answers
14k views

Dealing with correlated regressors

In a multiple linear regression with highly correlated regressors, what is the best strategy to use? Is it a legitimate approach to add the product of all the correlated regressors?
82
votes
9answers
60k views

Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 ...
97
votes
9answers
40k views

Is there an intuitive explanation why multicollinearity is a problem in linear regression?

The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...

1
16 17 18 19
20