# Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

993 questions
Filter by
Sorted by
Tagged with
4answers
2k views

### Multicollinearity and categorical predictor with three levels

If I have a continuous Dependent Variable and two Independent Variables, where one is categorical with three levels and the other is continuous, what assumptions do I need to check for multiple ...
1answer
658 views

### Does a decrease in standard error from one model to another suggest collinearity?

Say I have model 1 and model 2. y1 = b0 + b1x1 + u y2 = b0 + b1x1 + b2x2 + u If there is an increase in the standard error of x1 from model 1 to model 2, does this suggest collinearity between the ...
0answers
297 views

### Multicollinearity? Non-significant multiple-linear regression, highly correlated regressors but low variance inflation factor

I have a small sample (30 obs) and 4 independent variables, 3 of which are significantly correlated to each other. I have tried to run the simple linear regression on each of them separately and it ...
1answer
5k views

### Multicollinearity and interaction terms

My dataset has 2734 observations (but in some speficitations, that number reduces to 1280). I also have interaction terms (in some specifications, even fourth-order terms). As far as I know, ...
1answer
227 views

### How multicolinearity affect the prediction

In a linear regression model, if some of the predictors are correlated, then in the output of most software, you will see very large p-value in those coefficients and very high standard error. My ...
0answers
125 views

### Elastic net is being used in genome wide analysis. Similar approach would work for survey analysis?

I'm approaching the elastic net procedure for genome wide analysis (GWAS) because it allows for feature selection, groups detection and improved validity. It's a powerful technique when you have many ...
2answers
3k views

### How seriously should I consider the effects of multicollinearity in my regression model?

I have a model y ~ x + z and the correlation between x and z is 0.2. This is only weakly positive. So, how seriously should I consider the effects of ...
1answer
2k views

### Change in order of predictors breaks logistic model estimation (glm, R)

I am fitting a binomial logistic regression in R using glm. By chance, I have found out that if I change the order of my predictor variables, glm fails to estimate the model. The message I get is ...
1answer
15k views

### VIF for generalized linear model

Is the variance inflation factor useful for GLM models. Below example shows OLS is showing VIF>5, but GLM lower. GLM shows instability in the coefficients between train and test set. ...
1answer
62 views

### Multicollnearity in backward selection approach

When I build my most parsimonious model using a backward selection approach, do I have to worry about multicollinearity. I mean, do I first check for multicollnearity and drop the variables which has ...
2answers
2k views

### Multiple regression - how to deal with mixed linear and non-linear variables

Say I have a bunch of explanatory variables to predict a continuous independent variable. Below, a simple toy example: I think it would be easiest to do a log-log transform and proceed with linear ...
1answer
151 views

### Linear regression model mistakenly gives $R^2$ equal to 1

I'm using R to create a linear regression model from survey data about public sentiment for a new technology. I am encountering a problem where the addition of a new explanatory variable raises the ...
1answer
183 views

### transform variables then check for collinearity or other way round?

I was planning to construct a model with around 19 explanatory variables and one response variable. I have some confusion on following thing: If transformation is required in case of explanatory ...
1answer
284 views

### Multiple regression with correlated variables

I am trying to do multiple regression analysis of a continuous health variable (yval) keeping age, gender, height (cm), weight (kg) and waist (cm) as predictor variables for a database of about 7000 ...
1answer
71 views

### What to make of countervailing spatial regression coefficients?

I am running regressions across a country's counties (N about 300). I divide the country in two regions A and B to control for potential unobservables. My explanatory variable varies at the county ...
1answer
163 views

### Principal Component Regression with an additional factor

I am looking to tease out the significance and contribution of a particular variable to 2 different continuous responses. I have 7 continuous variables I know to be influential on the two responses (...
0answers
58 views

### Three percentage indicators into one (or two measures) in regression moder

I am working with a data set resembling the extract below: ...
2answers
1k views

### Invertibility of $X^TX$ with severe multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
3answers
7k views

### Collinearity and Linear Discriminant Analysis

I trying to conduct linear discriminant analysis using the lda package and I keep getting a warning message saying that the variables are collinear. I want to pinpoint and remove the redundant ...
3answers
542 views

### Collinearity in multivariate regression with huge amounts of data

Take the following example: I wish to predict physical performance as a function of height and weight. I already know weight negatively affects performance. Height also negatively affects performance,...
1answer
6k views

### Should we test error terms for auto correlation or multicollinearity

I understand the basic difference in definition between multicollinearity and autocorrelation. I.e multicollinearity describes a linear relationship between whereas autocorrelation describes ...
0answers
45 views

### Linear Regression - Number of Predictors - Negative Weight [duplicate]

I am trying to prep myself for data science interview & I saw the following question on a forum You have several variables that are positively correlated with your response, and you think ...
3answers
1k views

### Variance Inflation Factor less than 1 in ridge regression?

I was trying to determine the biasing constant in ridge regression when I came across a phenomenon that seems quite puzzling, to me at least. I let the GCV criterion choose a constant for me and then ...
2answers
2k views

### Predictor variables sum up to 1 but not necessarily correlated - is it a problem? [closed]

I am trying to fit hierarchical mixture model (using ML and MCMC, but this shouldn't matter) where the linear predictor part contains 17 independent variables. These are habitat variables: for each ...
0answers
1k views

### Principal component regression on polynomial terms

One of the data sets I am working upon had 3 variables which were having almost 100% correlation among themselves. Since I am learning regression modelling I thought I'll do principal component ...
0answers
1k views

### Should fixed effect dummies be included in multicollinearity diagnostics (VIF calculations) for regression?

I am currently in the progress of performing multicollinearity diagnostics for a logistic regression model using tolerance and VIF calculations based on recommendations in Allison (2012) (Logistic ...
1answer
777 views

### Why do we use the term multicollinearity, when the vectors representing two variables are never truly collinear?

When two vectors $a$ and $b$ are collinear, then $a = xb$, (where $x$ is a scalar) so in linear algebra, collinearity is a narrowly and clearly defined (and binary) concept. Two vectors -- in my ...
0answers
19 views

### Variable selection multiple data types

I'm working on a data set containing 45 predictors that cover metric, ordinal and nominal variables. Aiming to finally build a model, which explains the interacting effect of a set of predictor ...
1answer
7k views

### what if response variable is 'yes or no' in R?

How to analyze above the data to predict the probability that people have disease with a model? Factors thought to influence infection include city, age, and diet. BUT, I don't know how to do because ...
0answers
162 views

### Logistic principal component regression where PCs are correlated with an additional binary predictor

My scenario is this: I collected a bunch of vegetation data (% cover counts in a quadrant at different heights) in patches where birds were seen foraging and also in control patches where no foraging ...
1answer
846 views

### Covariate related to independent variable - best solutions

I have 2 groups (tinnitus sufferers and controls) who are significantly different in age - I would normally control for age as a covariate (as it is a cognitive task) but it violates the assumption of ...
1answer
1k views

0answers
1k views

### Looking for and dealing with collinearity in a GLM

I've got this dataset with one continuous dependent variable and two categorical explanatory variables. I'm wanting to run glms on the data but I'm finding problems with what I think is collinearity. ...
0answers
75 views

### Mean centering variables NOT in a moderation/interaction term [duplicate]

I am trying to assess the impact of multicollinearity in a regression because I have two separately measured variables which have the reversed signs problem (one predictor is +b regression weight, the ...
1answer
1k views

### Multicollinearity among categorical variables - Is it normal?

I'm building a logistic regression model in which almost all of the input variables are categorical. There are multiple sets of categorical variables, for example, day of the week, age range buckets, ...
3answers
153 views

### Too many predictors to manually check linearity

Say I have 1000 potential predictors in a logistic regression. I don't have time to check each predictor manually for linearity. I could wait till after variable selection, but in that case I wonder ...
0answers
1k views

### Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
5answers
13k views

### Coefficient changes sign when adding a variable in logistic regression

In my logistic regression the sign of coefficients of a variable (location distance of an amenity) changes based on other variables (with time -ve, with travel distance +ve) in the model. When the ...
1answer
661 views

### Curse of dimensionality mimics multicollinearity?

Why does the curse of dimensionality mimic multicollinearity, in the following sense.. Consider the random vector $Y = [y_{1}, \dots, y_{4}]$ where each element is ~ Uniform (0,1). Take 10 samples ...
2answers
1k views

### Many dependent variables, few samples: is this an example of “large $p$, small $n$” problem?

"Large $p$, small $n$" typically refers to "many independent variables, few samples". In my case, I have $1$ independent variable, $300$ dependent variables, and $n < 20$ samples. Thus, my case ...
1answer
2k views

### How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be non-...
2answers
1k views

### Importance of multiple linear regression assumptions when building predictive regression models

As far as I know, one can differentiate between two main goals of the regression analysis: The goal is understanding causal relations between variables. Here, one has to check several common ...