Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Why lasso is not a good idea when there are correlated covariates? [closed]

I saw some statements that if the covariates are correlated, then it is not a good idea to use Lasso. But I am wondering why this is the case?
1
vote
0answers
16 views

Combating Multicollinearity in Multivariate Granger Causality

Let's say that I have a variable $X_1$ that Granger-Causes a variable $Y$. For simplicity, assume only 1 lag. Now, let's say I have two more variables $X_2$ and $X_3$ which happen to be exact copies ...
11
votes
1answer
201 views

Why is multicollinearity different than correlation?

I know that someone will probably say that this question is repeated and I will get a negative vote, but I'm very convinced that it's not, or at least it wasn't properly answered. See, we have lots of ...
0
votes
0answers
22 views

Does the null space of the design matrix X have an interpretation in statistics?

I am going through the material I learned in my linear algebra classes and I am trying to view the material from a statistical perspective. This typically boils down to imagining what the theorems say ...
0
votes
0answers
18 views

Logistic Regression inversely proportional correlation [duplicate]

This is more of a clarification question than anything else, but I am familiar that when dealing with regression algorithms it is useful to eliminate features that are highly correlated. I have been ...
0
votes
0answers
8 views

Variable relationship case: x1 is not correlated to y but x2 which is related to x1 is correlated to y

I would like to know what is the specific term given to this case, or how is this justified in technical words: x1 is not correlated to y but x2 which is correlated to x1 is correlated to y.
1
vote
0answers
28 views

Categorical features driven perfect collinearity in logistic regression [duplicate]

In OLS regression, when one has a set of sum-to-one dummies one needs to take one out to avoid perfect collinearity (be able to invert X^TX), in logistic regression as far as I understand one does not ...
1
vote
1answer
23 views

can I switch one collinear predictor onto dependent variable position in a model?

I have a situation where my current model looks like MODEL 1: C ~ A * B + (1|random effect, that is C is predicted by ...
0
votes
0answers
13 views

Permutation Importance with Multicollinear Features Confusion

sklearn have a very interesting article, in which they explain a procedure to handle multicollinearity between features when assessing feature importance by: ...
0
votes
0answers
16 views

Checking fixed effects regression assumptions

I have a panel data set which I have fit a fixed effects model to using plm() in R. The Hausman test indicated that a fixed effects model should be used over a ...
0
votes
1answer
23 views

When using variance inflation factor (VIF) should I do the removal recursively?

This is a pretty straightforward question and I guess I will get a negative score here - I was so happy improving my points in here lol -, but I couldn't find anywhere and even though I believe I know ...
1
vote
1answer
28 views

OLSR model: high negative correlation between 2 predictors but low vif - which one decides if there is multicollinearity?

Date, age, mrt and shops are all predictors in a dataset of 414 observations. Pearson's product-moment correlation shows a sizeable negative correlation between mrt and shops (-0.6 so definitely ...
1
vote
0answers
26 views

Strongly and weakly correlated variables and PCA analysis in a prediction problem?

I am doing a prediction assignment as part of a machine learning course using loans data. I have just done some exploratory data analysis on my dataset of just over 9000 rows. There are 11 variables ...
0
votes
0answers
21 views

What do you do when a random effect is collinear with a fixed effect?

Say I have a dataset which has rows of observations of Mood (HAPPY/SAD), with info on the <...
2
votes
0answers
18 views

Where is multicollinearity when we talk about LINE assumptions?

I was reading about Linear Regression assumptions and I'm a little confused cause lots of websites and books mention different assumptions. What I found to be the most common assumptions mentioned are ...
0
votes
0answers
45 views

Regression when predictors are correlated by design (single measurement vs. long-term average)

I am interested in determining if a biological response variable is more strongly related to the value of an environmental predictor variable measured the same year in which the response was measured, ...
0
votes
0answers
4 views

when we say correlation is 1 in boss Ridge and Elastic Net, does it only mean $x_1 = x_2$ for the allocation of weights

Question: when we say correlation is 1 in boss Ridge and Elastic Net, does it only mean $x_1 = x_2?$ Story: Ridge will trends to allocate the similar coefficients ...
0
votes
0answers
31 views

How does GLM in R behave when given highly correlated predictors [duplicate]

I tried to use GLM in R, the y value is given below y <- c(2.0875796, 1.0857121, 0.2783329, 0.7866724, 1.7395036, 1.6341974, 0.1919819, -0.9013408, -1.1337154, 0.4611232, 2.1412645, 2.0390984, 1....
0
votes
0answers
9 views

Testing for multicollinearity: use same specification as main model and clustered standard errors or not

When I am trying to check multiple panel data for multicollinearity by regressing one independent variable on another, should I use exactly the same specification as the main model or is an OLS ...
1
vote
2answers
47 views

Standardize one predictor variable or all predictor variables to solve multi-collinearity

I was using a fixed-effects panel model with interaction effects when I realized that the VIF values are too high for some variables. I was advised to standardize the predictor variables to mitigate ...
6
votes
2answers
95 views

Is there any gain by adding correlated categorical variables (e.g.: city and neighborhood) in a model?

It's a very straightforward question that I've never seen any discussion. To keep it simple, let's say I have a Linear Regression and I want to predict housing prices. I have a dataset that contains ...
1
vote
1answer
20 views

Lollipop plot in R with Ridge Regression coefficients [closed]

I have the following dataset, in which I want to understand the influence of four explanatory variables (X1, X2, ...
0
votes
1answer
40 views

Ridge Regression Graph

I intend to assess how some variables relate to a particular response variable: ...
0
votes
0answers
16 views

Alternative for multicollinearity problem

I am a bit unsure how to specify my model and therefore look for some references / ideas. I am interested in a model along the following lines: $pay_i = \alpha + \gamma X_i + \beta pop_i + \epsilon_i ...
1
vote
0answers
32 views

VIF (multicollinearity), Breusch-Godfrey (autocorrelation), Breusch-Pagan (heteroskedasticity) for Linear Regression

We are conducting linear regression. We performed first the Variance Inflation Factor to check for multicollinearity and we dropped the independent variables below 10 So are ALL of our independent ...
0
votes
1answer
15 views

Joint Hypothesis Testing on Multicollinear Regressors?

I was reading the following thread: link when I came across this discussion regarding dropping regressors that demonstrate multicollinearity from a linear regression model: But what if you have ...
0
votes
0answers
40 views

How to explain the phenomenon that each coefficient is significant in multiple regression but not significant as simple regression [duplicate]

We know that in linear regression, when each coefficient is not significant in multiple regression but significant as a simple regression, it is most likely the reason of ...
0
votes
0answers
10 views

Writing OLS estimates as a function of real parameters in case of multivariate regression

Suppose that in the population: $$ y = \alpha + \beta_1 x_1 + \epsilon $$ We now estimate the model: $$ \hat{y} = \hat{\alpha} + \hat{\beta_1} x_1 $$ If we try to estimate $\beta$ using OLS, we have ...
0
votes
0answers
53 views

Include exogenous variable in both level and first-difference

I am wondering if I could include, in a VECM, exogenous variable in both level and first-difference, such as : $\Delta X_t = \alpha \beta^{'} X_{t-1} + \Delta X_{t-1} + Y_t + \Delta Y_t + U_t$ Is ...
0
votes
0answers
23 views

Multicollinearity in quadratic (polynomial) regression function [duplicate]

Multicollinearity problem could arise when we add quadratic variable in regression like this: So, one of the possible solutions to eliminate the problem is to add centered variables: This was ...
0
votes
0answers
35 views

Which machine learning model is reliable when my data face multicollinearity issue?

I have 27 features and I'm trying to predict continuous values. When I calculated the VIF (VarianceInflation Factors), only 8 features are less than 10 and the remaining features range from 10 to 250. ...
0
votes
0answers
20 views

How do you make dummies for collinear categorical variables?

Suppose I have the following relationship $$y = A + B + \epsilon$$ where $A$ is a categorical variable with $2$ levels and $B$ is categorical with $3$ levels. However when $A = 1$, $B=1$, always. How ...
3
votes
0answers
50 views

Does it make sense to deal with multicollinearity prior to LASSO regression?

Does it ever make sense to check for multicollinearity and perhaps remove highly correlated variables from your dataset prior to running LASSO regression to perform feature selection? One of the ...
0
votes
2answers
30 views

Variance Explained by a Large Number of (Colinear) Variables

Currently working on a project that explores how collectively, 121 variables about the environment, predict a single outcome. We run into two major issues: Our variables are highly colinear. Rainfall ...
0
votes
0answers
20 views

Is there a way to combine features there is correlated but is important(time series) in a classification problem?

Suppose I have variables indicating access by months, consults by months (and other variables) and I want to predict the digital propension. Digital_acess_mont01 Digital_acess_mont02 ...
0
votes
1answer
34 views

Will we use ridge in linear regression if there is no multicolinearity

I know that adding L2 regularization (ridge) can reduce multicolinearity in linear regression. I originally understand as multicolinearity will increase the estimation variance and L2 regularization ...
2
votes
1answer
13 views

Instrument validity: does a positive and significant coefficient on Z in a regression of Y on X and Z pose a problem?

I have an initial regression of Y on X and Z. Both of my coefficients on X and Z are non-zero and strongly statistically significant. X and Z are correlated but I am told collinearity shouldn't be an ...
2
votes
0answers
18 views

Two-Step Procedure to account for multicollinearity

Suppose the estimation equation is $$ y=\beta _{0}+\beta _{1}x_{1}+\beta _{2}x_{2}+\varepsilon $$ where $\varepsilon $ is a disturbance and $x_{1}$ and $x_{2}$ are highly correlated regressors. In a ...
3
votes
3answers
188 views

Regression Performance Measures: Alternatives to MSE

Preface I am dealing with five Regression models (Ordinary Least square, Least Absolute regression, Huber Regression, MM Estimator, and Ridge Regression). I try to check which model is more robust to ...
2
votes
2answers
151 views

How can I deal with a covariate being defined only for a subset of my sample?

My model looks like: ROI_size ~ diagnosis + medication_dose + sex + age. Specifically, I want to find the effect of disease (1 or 0), adjusted for current medication dose (measured in mg) on brain ...
0
votes
1answer
15 views

Dependence between votes

I'm having regions and votes in those regions for different parties for two different elections. Is there a way to identify whether some party took the voters of another party in some region. I can ...
0
votes
0answers
11 views

How to fix multi-collinearity in levels of a categorical variable?

I am running a bayesian logistic regression in rstanarm and bayestestR in R and part of the results return the following: ...
2
votes
0answers
68 views

"The matrix is either rank-deficient or indefinite" error in fixed effects linear model (felm)

I estimate various felms in R, which worked fine until I added covariates from t(the year of the merger)-2. I estimate the effects of a merger on RD expenses in t till t+3, and have covariates of RD ...
0
votes
0answers
9 views

Do I need to generate orthogonal polynomial in scikit learn for logistic regression classifier?

I tried to make a logistic regression classifier to predict the class of unseen data. In this data cross terms play important role to predict the multi imbalance class, with this information I cannot ...
0
votes
1answer
49 views

what exactly does correlation mean [duplicate]

what exactly is a correlation of zero meant to be, does that mean that changes in x does not affect y, if that is the case, from the scatter plot That is completely false, as sometimes as x increases,...
1
vote
1answer
62 views

How do I eliminate linear dependence in a difference-in-difference model in R?

I am using panel data to try and observe the effects of Vietnamese immigration in the California Bay Counties in the 1980s. I am using R. I am regressing average weekly wages (adjusted for inflation) ...
0
votes
1answer
30 views

Does colinearity between independent factors weaken my model?

I understand if a linear regression model is used colinearity is a major problem and thus my model can be weak. However, what is the case if I got a trained model with high R-squared from other ...
0
votes
0answers
20 views

Problem understanding why a variable of interest becomes significant when adding another variable

Now this is a thoroughly discussed topic but unfortunately I've never come across an explanation that is intuitive, also there may be several reasons, none of which are intuitive. I have a study in ...
0
votes
1answer
14 views

I am doing a simple Moderation regression: there is multicolinearity in my model when I add the interaction term. I want to check if this is ok

The first step of my regressionmodel with both predictor variables and the outcome variable meet all assumptions. However when I add the second term there is multicolinearity. This seems obvious since ...
1
vote
2answers
126 views

Why multicollinearity increases with country fixed effects in linear model in r

I'm playing with some multiple linear regression models in r. After I run a regression, I use vif() to see if there is multicollinearity between my predictors. For ...

1
2 3 4 5
21