Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

0
votes
0answers
5 views

Proof for multicollinearity consequence

I came across this statment "Even extreme multicollinearity (so long as it is not perfect) does not violate OLS assumptions. OLS estimates are still unbiased and BLUE (Best Linear Unbiased Estimators)"...
0
votes
0answers
13 views

Interpreting $p$-values and Chi square when variables are added to a model in maximum likelihood estimation

I'm running a maximum likelihood estimation (probit) and I'm experimenting with adding additional variables, walking the bias-collinearity tightrope. Please could someone explain to me intuitively ...
0
votes
1answer
25 views

solutions for muticolinearity

how do you linearly combine predictors, such as adding them together to avoid multicollinearity? I have multicolinearity in my regression. say the function is lm(y ~ x + z) where x and z have ...
0
votes
0answers
5 views

Is repeated propensity score matching over many 0-1-features a valid procedure?

I would like to do a simple linear model where the outcome $y$ is real-valued, but my data matrix $X$ consists of several hundred features that all are $0$-$1$-valued. The number of observations $n$ ...
1
vote
2answers
22 views

Estimating effect of linear regression coefficients with multicollinearity

As I didn't find a satisfying for that questions I try it here: I have a multivariate Lineare Regression model with some correlated predictor variables. The "simple" question I want to answer is: "If ...
1
vote
0answers
22 views

Can a complex interaction term mean more than what it's composed of?

I'm cross-posting this question on both Economics and Cross Validated to get answers from a different perspective on each field. It is generally accepted to cross-post if the question is tailored to ...
1
vote
1answer
31 views

multicollinearity between confounders logistic regression

Im going to investigate if a disease have an negative impact on a binary response variable. The disease is the independent variable with additionally confounders. I want to do a manual stepwise ...
0
votes
1answer
18 views

Non-significant in correlation, but significant predictor in regression. How to explain suppression?

I'm having trouble explaining some results... I have 5 independent variables (A, B, C, D and E) and I want to know their relation to the dependent (Y). Only variables A and C are significantly ...
6
votes
4answers
157 views

Can i include the product of two random variables? Or do I risk collinearity?

I have a model in which I want to predict Y. My regressors X, are x1 and x2. For some reason I believe that it would also be useful to include into the model: ...
2
votes
1answer
56 views

Construction of linear mixed model (using R)

I would like to use Lineal Mixed model to see if the treatments I applied to some soil changed significantly their CO2 fluxes. I have 2 temperature (t1, t2) and 3 inundation (w0,w1,w2), resulting in ...
0
votes
0answers
32 views

Doesn't high feature correlation decrease random forest accuracy?

I have generated a dataset of artificial data and want to distinguish two labels from each other using a random forest. I thought having correlated features in my dataset will decrease the algorithms ...
0
votes
2answers
30 views

Why is my variable being omitted by stata?

I am carrying out a fixed effect regression. I have a dummy variable called female. The dependent variable docvis refers to hospital visits. I created an interaction term between hhkids and female ...
1
vote
1answer
30 views

Use Residuals to remove multicollinearity

This is probably a matter of me not knowing the terminology, but suppose I want to isolate the effect of $X$ on $Y$, and I have some other factor $Z$ that is somewhat correlated with $X$. So the ...
0
votes
0answers
11 views

How to estimate the VIF for geeglm models in r

I am very new in r and at analyzing gee models. I have a very high dimensional data (51 independent variables measure at multiple times with no missing values (secondary dataset)). I am pretty sure ...
0
votes
0answers
13 views

OLS Multicolleniarity

I have a pretty simple task to estimate ols multiple regression. I need a measure of multicolleniarity. Is condition number a good measure and what criteria exists fot its value?
0
votes
0answers
73 views

Error in vif.default(glm.fit1) : there are aliased coefficients in the model

I have 12,000 records and I'd like to predict a two-class outcome. The dataset has 3 numeric predictors and twenty categoric predictors. The problem is that I have perfect collinearity somewhere ...
0
votes
0answers
19 views

How do I diagnose collinearity with rfe() from the caret package?

I have 12,000 records and I"d like to predict a two-class outcome. I'm deciding which predictors to keep and I'm having trouble with two problems. 1- I get an error message because I have categories ...
1
vote
1answer
30 views

Create composite variables using items with different scales- dealing with multicollinearity

I have IV's that are highly correlated with one another. The first set of correlated IV's, I combined them by adding the score and dividing by 2 to create a composite score. This was simple because ...
2
votes
0answers
77 views

Interpreting interaction term on highly correlated variables

Somebody at a meeting today made the following comment about a Marketing Mix Model (Linear Regression) we run every year. We should account for the high collinearity of the two Marketing Variables (...
0
votes
3answers
65 views

Why should I check for collinearity in a linear regression?

The Gauss-Markov Assumptions: MLR.1: Linearity in parameters. MLR.2: Random sampling. MLR.3: No perfect multicollinearity. MLR.4: Zero conditional mean Hence, why should I check for high (but not ...
0
votes
0answers
22 views

Is it ever okay to ignore a low Cramer's V value?

I conducted a survey on college students to assess what demographic variables affected their views of and knowledge towards climate change. There are 11 variables total: Gender Religion Political ...
0
votes
0answers
52 views

How to test multicollinearity in multinomial logistic regression when we have both continuous variables as well categorical variables as predictors?

How to test multicollinearity in multinomil logistic regression? I have 25 independent variables and 1 dependent variable. Out of 25 independents variables, 17 variables are continuous variables and 8 ...
0
votes
0answers
101 views

GVIF output in R

After fitting a logistic regression model m I've run vif() on it and been given the following output ...
0
votes
0answers
15 views

Generating correlated data using numpy while controlling multicollinearity

I am using the following code (adopted from the code in this post). I have no problems with the code. My question is that if with this code I can create or prevent multicollinearity among the ...
0
votes
0answers
13 views

Removing multi-colinearity by running pair-wire regression and only take the residual?

Given a multi regression problem: y ~ $\beta_{1}x_{1}$ + $\beta_{2}x_{2}$ + $\beta_{3}x_{3}$ + $\beta_{4}x_{4}$ ...$\beta_{n}x_{n}$ If we found that x2 and x3 are linearly correlated, which will ...
1
vote
0answers
35 views

Endogeneity versus multicolinearity in regressions?

(I limit myself to two explanatory variables to keep it simple) So as I understand it, when an explanatory variable is highly correlated with another variable then it becomes difficult to distinguish ...
3
votes
2answers
79 views

Omitted Variable Bias & Multicollinearity: Why are the coefficient SEs smaller in the unbiased specification?

In Introductory Econometrics: A Modern Approach, Wooldridge writes the following regarding the omitted variable bias and its effect on the variance of the OLS estimator (x1 and x2 are correlated): ...
1
vote
0answers
27 views

Dealing up with collinearity predictors' choice in xreg using auto.arima

I'm trying to do a regression with arima errors in R, with xreg in auto.arima following https://otexts.com/fpp2/ by https://robjhyndman.com/ but I have some questions about the predictors' choice in ...
3
votes
3answers
62 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
1
vote
0answers
69 views

Understanding VIF procedure with different methods from r package rms and car - VIF by level of a factor

Consider the output from two different r packages, rms::vif and car::vif: ...
1
vote
5answers
133 views

How do I handle terms with collinearity?

I'm working on a regression model with the Hitters data from the ISLR package. It has ~300 observations and 20 variables. I want to predict a player's salary. I have major problems of collinearity ...
0
votes
1answer
53 views

I am making a logistic regression model. Should I test for multicollinearity in dependent features if my predicting feature in categorical?

I have a doubt, will multicollinearity affect my Logistic Regression model as my predicting(output) feature is categorical? (because correlation will make sense only for 2 continuous features and not ...
0
votes
0answers
9 views

how can I interpret a long-run covariance matrix?

can I use the long-run covariance matrix to examine multicollinearity between my independent variables? and if yes, what how can I interpret matrix for determining Multicollinearity?
1
vote
0answers
26 views

What is the meaning of correlations between regression coefficients?

I understand that coefficient correlations can arise in the presence of correlated covariates, essentially indicating that our inference for those parameters is coming from information that cannot be ...
0
votes
0answers
11 views

vif range in multi linear regression model

Explain the interpretation of the result of vif .I have 3 independent variables in a multi regression model and the all the three vif values are less than 3.What interepretations I can draw from the ...
-1
votes
1answer
38 views

multicollinarity issue

I ran a logistic regression moderation analyses and I noticed that with the addition of the interaction term, one of the predictors flipped signs. The variable (X) was previously -.015 and became .006 ...
1
vote
0answers
25 views

Uniqueness of partial covariance/corrlation if OLS is not unique

Let $X,Y,Z=(Z_1,...,Z_n)$ be random variables. Define the partial covariance between $X$ and $Y$ given $Z$ as: $$\rho_{X,Y \cdot Z} := cov(\hat{X}-X, \hat{Y}-Y)$$ where $\hat{X}$ and $ \hat{Y}$ are ...
1
vote
0answers
41 views

Interpretation of parameter stability of regression coefficients and stability of predictions when multicollinearity is present

It is known that when the predictor variables are highly correlated with each other (e.g, correlation coefficient is 0.9) the regression coefficients are unstable as they have high standard errors. I ...
0
votes
0answers
150 views

Why is the value of vif from $(X'X)^{-1}$ not matching the result?

The diagonal elements of matrix $C = (X'X)^{-1}$ are $C_{jj} = \frac{1}{1-R_{j}^{2}}$ (which is nothing but the vif) of $x_j$ where $j = 1, 2, 3, ..., n$ and $X$ is a $n\times p$ matrix and $R_j^2$ ...
0
votes
0answers
34 views

Does standard error affected by the coefficient?

I make a comparison on ridge regression and OLS using simulation. As i set my correlation as 0.9, which is high, i expect the standard error of ridge regression to be low. However, it is not. ...
0
votes
0answers
78 views

Cox proportional hazards model, problem with correlated predictors or overfitting?

I have a question concerning a Cox proportional hazard model (in R) on which I would love to get your opinion and feed back! I think that there is a problem, however I would like to be sure and ...
0
votes
0answers
33 views

Softmax regression vector features?

I'm working on a difficult classification problem where there are a very large number of known classes (~50k). I have only about 20k labelled points, but these only represent <1% of the possible ...
1
vote
0answers
53 views

R code for robust ridge regression

I am having trouble in searching for the MSE value in using robust ridge regression. The robust estimators that i used is LTS and MM. However, when both robust estimators were applied to ridge, the ...
0
votes
0answers
21 views

Incorporating prior knowledge into feature selection in the setting of multicollinearity?

Background: I'm trying to find the optimal combination of two parameters for finding the first peak meeting some criteria in a signal. The filtering is a bit simplistic, there's a threshold (...
4
votes
3answers
401 views

Explaining multicollinearity in layman's terms

Say we have a study where we want to run a logistic regression on a group of people, and we want to find out whether one attribute of a person makes them more likely to be a smoker. So we have smoker ...
0
votes
0answers
10 views

How does Stata test for collinearity? [duplicate]

My question is what stata is doing behind the scenes to detect collinearity. I previously assumed that it simply checked to see if there was a strict linear relationship between two variables, as in: ...
1
vote
1answer
51 views

Is this a valid work-around for collinearity?

A fellow PhD student has monthly data on temperatures (T) and precipitation levels (P) for a certain agricultural region. He would like to use it to predict total farm revenues (Y) for year t: $Y_t=\...
0
votes
0answers
14 views

Linear Regression on Boston Housing Price? [duplicate]

As far my knowledge, Linear Regression assumes that data or columns are normally distributed and doesn’t have multicollinearity amongs the features, But when I apply Shapiro test, it shows that none ...
1
vote
1answer
50 views

How can I use linear/logistic regression for inference with colinear variables and a smallish dataset?

I have a dataset of around 120 observations, with 30 calculated variables and I am trying to predict a continuous response (result of an experiment) using those 30 variables. Ideally the smallest ...
0
votes
0answers
21 views

Steps after calculating VIF to deal with multi collinearity

How do we eliminate variables after calculating VIF ? I have read about step wise removal of variables with high VIF till we reach VIF values below 5. On the other hand , my professor told me that ...