# Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

1,043 questions
Filter by
Sorted by
Tagged with
10 views

### Why lasso is not a good idea when there are correlated covariates? [closed]

I saw some statements that if the covariates are correlated, then it is not a good idea to use Lasso. But I am wondering why this is the case?
16 views

### Combating Multicollinearity in Multivariate Granger Causality

Let's say that I have a variable $X_1$ that Granger-Causes a variable $Y$. For simplicity, assume only 1 lag. Now, let's say I have two more variables $X_2$ and $X_3$ which happen to be exact copies ...
201 views

### Why is multicollinearity different than correlation?

I know that someone will probably say that this question is repeated and I will get a negative vote, but I'm very convinced that it's not, or at least it wasn't properly answered. See, we have lots of ...
22 views

### Does the null space of the design matrix X have an interpretation in statistics?

I am going through the material I learned in my linear algebra classes and I am trying to view the material from a statistical perspective. This typically boils down to imagining what the theorems say ...
18 views

### Logistic Regression inversely proportional correlation [duplicate]

This is more of a clarification question than anything else, but I am familiar that when dealing with regression algorithms it is useful to eliminate features that are highly correlated. I have been ...
8 views

### Variable relationship case: x1 is not correlated to y but x2 which is related to x1 is correlated to y

I would like to know what is the specific term given to this case, or how is this justified in technical words: x1 is not correlated to y but x2 which is correlated to x1 is correlated to y.
28 views

### Categorical features driven perfect collinearity in logistic regression [duplicate]

In OLS regression, when one has a set of sum-to-one dummies one needs to take one out to avoid perfect collinearity (be able to invert X^TX), in logistic regression as far as I understand one does not ...
23 views

### can I switch one collinear predictor onto dependent variable position in a model?

I have a situation where my current model looks like MODEL 1: C ~ A * B + (1|random effect, that is C is predicted by ...
13 views

### Permutation Importance with Multicollinear Features Confusion

sklearn have a very interesting article, in which they explain a procedure to handle multicollinearity between features when assessing feature importance by: ...
16 views

### Checking fixed effects regression assumptions

I have a panel data set which I have fit a fixed effects model to using plm() in R. The Hausman test indicated that a fixed effects model should be used over a ...
23 views

### When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
28 views

### OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
26 views

### Strongly and weakly correlated variables and PCA analysis in a prediction problem?

I am doing a prediction assignment as part of a machine learning course using loans data. I have just done some exploratory data analysis on my dataset of just over 9000 rows. There are 11 variables ...
21 views

### What do you do when a random effect is collinear with a fixed effect?

Say I have a dataset which has rows of observations of Mood (HAPPY/SAD), with info on the <...
18 views

### Where is multicollinearity when we talk about LINE assumptions?

I was reading about Linear Regression assumptions and I'm a little confused cause lots of websites and books mention different assumptions. What I found to be the most common assumptions mentioned are ...
45 views

### Regression when predictors are correlated by design (single measurement vs. long-term average)

I am interested in determining if a biological response variable is more strongly related to the value of an environmental predictor variable measured the same year in which the response was measured, ...
4 views

### when we say correlation is 1 in boss Ridge and Elastic Net, does it only mean $x_1 = x_2$ for the allocation of weights

Question: when we say correlation is 1 in boss Ridge and Elastic Net, does it only mean $x_1 = x_2?$ Story: Ridge will trends to allocate the similar coefficients ...
31 views

### How does GLM in R behave when given highly correlated predictors [duplicate]

I tried to use GLM in R, the y value is given below y <- c(2.0875796, 1.0857121, 0.2783329, 0.7866724, 1.7395036, 1.6341974, 0.1919819, -0.9013408, -1.1337154, 0.4611232, 2.1412645, 2.0390984, 1....
9 views

### Testing for multicollinearity: use same specification as main model and clustered standard errors or not

When I am trying to check multiple panel data for multicollinearity by regressing one independent variable on another, should I use exactly the same specification as the main model or is an OLS ...
47 views

### Standardize one predictor variable or all predictor variables to solve multi-collinearity

I was using a fixed-effects panel model with interaction effects when I realized that the VIF values are too high for some variables. I was advised to standardize the predictor variables to mitigate ...
95 views

### Is there any gain by adding correlated categorical variables (e.g.: city and neighborhood) in a model?

It's a very straightforward question that I've never seen any discussion. To keep it simple, let's say I have a Linear Regression and I want to predict housing prices. I have a dataset that contains ...
20 views

### Lollipop plot in R with Ridge Regression coefficients [closed]

I have the following dataset, in which I want to understand the influence of four explanatory variables (X1, X2, ...
40 views

### Ridge Regression Graph

I intend to assess how some variables relate to a particular response variable: ...
16 views

188 views

### Regression Performance Measures: Alternatives to MSE

Preface I am dealing with five Regression models (Ordinary Least square, Least Absolute regression, Huber Regression, MM Estimator, and Ridge Regression). I try to check which model is more robust to ...
151 views

### How can I deal with a covariate being defined only for a subset of my sample?

My model looks like: ROI_size ~ diagnosis + medication_dose + sex + age. Specifically, I want to find the effect of disease (1 or 0), adjusted for current medication dose (measured in mg) on brain ...
15 views

I'm having regions and votes in those regions for different parties for two different elections. Is there a way to identify whether some party took the voters of another party in some region. I can ...
11 views

### How to fix multi-collinearity in levels of a categorical variable?

I am running a bayesian logistic regression in rstanarm and bayestestR in R and part of the results return the following: ...
68 views

### "The matrix is either rank-deficient or indefinite" error in fixed effects linear model (felm)

I estimate various felms in R, which worked fine until I added covariates from t(the year of the merger)-2. I estimate the effects of a merger on RD expenses in t till t+3, and have covariates of RD ...
9 views

### Do I need to generate orthogonal polynomial in scikit learn for logistic regression classifier?

I tried to make a logistic regression classifier to predict the class of unseen data. In this data cross terms play important role to predict the multi imbalance class, with this information I cannot ...
49 views

### what exactly does correlation mean [duplicate]

what exactly is a correlation of zero meant to be, does that mean that changes in x does not affect y, if that is the case, from the scatter plot That is completely false, as sometimes as x increases,...
62 views

### How do I eliminate linear dependence in a difference-in-difference model in R?

I am using panel data to try and observe the effects of Vietnamese immigration in the California Bay Counties in the 1980s. I am using R. I am regressing average weekly wages (adjusted for inflation) ...
30 views

### Does colinearity between independent factors weaken my model?

I understand if a linear regression model is used colinearity is a major problem and thus my model can be weak. However, what is the case if I got a trained model with high R-squared from other ...
20 views

### Problem understanding why a variable of interest becomes significant when adding another variable

Now this is a thoroughly discussed topic but unfortunately I've never come across an explanation that is intuitive, also there may be several reasons, none of which are intuitive. I have a study in ...