Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
1 vote
1 answer
33 views

Does multicollinearity among control variables matter?

I am conducting a regression analysis between X and Y, where X is the main independent variable. However, I want to control for several variables that are related to Y. For example if my dependent ...
  • 11
0 votes
0 answers
34 views

Inclusion of time-related variable in longitudinal mixed model

I will use the sleepstudy dataset from the lme4 package to illustrate my question. In that dataset, the Reaction time of individual Subjects was measured each Day. Subjects were sleep deprived so as ...
0 votes
0 answers
3 views

Does adding features from "persistent homology" of the data increase multicollinearity, condition numbers of hessian, and endogenous variable?

Does adding features the "persistent homology" of the data increase multicollinearity, condition numbers of hessian, and endogenous variable? The persistent homology is a summary statistic ...
1 vote
1 answer
19 views

Controlling for both time fixed effects and "entity invariant" variables

I have a question about a panel data regression model in general. I have firm-level data in a single country. Is it okay to control for both time fixed effects and entity-invariant variables, such as ...
  • 13
0 votes
2 answers
52 views

MLE to address multicollinearity in linear regression

OLS estimation assumes that the explanatory variables are independent in the linear regression model. There isn't such assumption when using the MLE estimation. So, my question is, can we use MLE to ...
  • 11
0 votes
0 answers
5 views

Two variables when added give third variable in a binary classification dataset. Should I remove one of them?

I have a dataset with over 90 variables. The target variable is a binary categorical variable. There are three particular (continuous) variables in the dataset, say a, b, and c. I have observed that a ...
1 vote
1 answer
105 views

Elastic Net Collinearity

When performing linear regression it is often assumed that the predictors are independent with Gaussian noise: \begin{equation} Y = X\beta + \epsilon \quad \epsilon \sim \mathcal{N}(0, \sigma) \end{...
  • 121
0 votes
0 answers
19 views

identifying the multicollinearity when Intercept has a very high variance

I applied the multinomial logistic regression and use the following function colldiag to calculate the Multicolinearity. It seems bottom line (index=39) shows ...
  • 13
1 vote
0 answers
38 views

Univariate Regression Coefficients and Multivariate Regression Coefficients

I got the following question: Suppose I have three variables, $x_1$, $x_2$ and $y$. We run univariate regression of $y$ on $x_1$ ($x_2$) with intercept and get the regression coefficients $\beta_1$ ($\...
  • 213
1 vote
0 answers
15 views

Independent variables affected by size, how I resolve this problem?

I am doing a research about the impact of trade unions on permanent contracts. The dependent variable is trade union presence (0,1), the independent variable is number of permanent contracts ...
0 votes
1 answer
56 views

How is this simple event study design collinear?

I am trying to estimate a very simple model in the form: $Y_{it} = a'D_{event} + b'D_{i} + c'D_{t} + e_{it}$ With the outcome $Y$, the individual fixed effects $D_{i}$, calendar year fixed effects $D_{...
  • 11
1 vote
0 answers
30 views

Understanding feature importance for collinear features with tree-based models

I'm trying to understand how collinearity affects feature importance for tree-based models. My understanding is that tree-based models naturally overcome multicollinearity for the purposes of ...
0 votes
1 answer
21 views

what does it means if two variable with exactly the same Wald Chi-Square?

After I run the program, it shows two variables with exactly the same Wald Chi-Square, how can I interpret these statistics? Can I say that it is Quasi-complete separation that a single level of a ...
1 vote
0 answers
43 views

Potential problem with multicollinearity?

I am trying to figure out what happens to my results if I unintentionally introduced multicollinearity. I have an unadjusted version of this regression that includes an interaction term. The ...
1 vote
1 answer
29 views

How to generate variables with common latent factor?

I'm dealing with Kalnins (2018) work about multicollinearity. I can't comprehend one thing - how can I get variables that share a common latent factor? (not only correlate with each other). I can ...
0 votes
0 answers
6 views

Can I use partial least squares regression in Baron and Kenny's mediation model?

I am writing my dissertation on self-esteem as a mediating variable between parenting styles and academic procrastination. I included measures of Baumrind's parenting styles, including both maternal ...
4 votes
1 answer
80 views

Handling multicollinearity with Restricted Least Squares

The dummy variable trap - including a dummy variable for every category and including a constant term in the regression together guarantees perfect multicollinearity - is most commonly resolved by ...
  • 73
0 votes
0 answers
29 views

Estimates of correlated predictors and R squared in a multiple linear regression model

I am currently working out how different predictors contribute to a multiple linear regression model, especially when they are correlated and how it effects $R^2$. Given this diagram... the author ...
  • 1
2 votes
2 answers
153 views

What to do when every VIF value is infinity

I know that VIF values have no upper limit, and that anything over 10 is usually bad news if you are trying to avoid multicollinearity especially for regression models such as multiple logistic ...
0 votes
0 answers
19 views

Fixed-Effects Correlated with Another Dummy

Question regarding convention in econometrics. Suppose I am running a two-way Fixed Effect (FE) model on a District-Year panel. Year varies from 2000 to 2010. Suppose I want to examine the effect of ...
1 vote
1 answer
23 views

High correlation between predictor and outcome

I am fitting a linear model which is trying to predict a certain quantitative variable (volume after treatment). I am trying to make inference in which other variables influences this volume. One of ...
1 vote
0 answers
49 views

interpretation of estimate concurvity vs worst concurvity in GAMs

I was wondering if there is someone who could explain (in conceptual terms) how to interpret estimate concurvity in a GAM implemented with mgcv and how it differs from worst concurvity (as obtained ...
  • 139
0 votes
1 answer
26 views

Linear Regression using aggregate variable create multicollinearity or adjustment issue?

my group project has a number of independent binary variables x1,x2,...,xm and a dependent binary variable y. My dataset contains some million of rows. Since there are so many independent variables, ...
0 votes
1 answer
33 views

"NA" in multiple regression

I have a question about multiple regression data$x4 <- with(data, x1+x2) #x4=x1+x2 y = b0 + b1x1 + b2x2 + b3x3 mod1 <- y ~ x1 + x2 + x3 + x4 I ran ...
  • 1
1 vote
0 answers
35 views

Batchwise Variance Inflation Factor Calculation for Feature Selection

I am currently working on a research project that involves interpretation of coefficients from a fitted Cox Proportional Hazards model. Unfortunately, the features within my model exhibit high ...
0 votes
0 answers
32 views

Multiple Regression - VIF

could anyone help me with my problem - I’m a beginner. When doing multiple regression and checking for multicollinearity, I have found 2 VIF’s of 17, however, I’ve done some reading and few say it is ...
1 vote
1 answer
79 views

How to solve perfect collinearity with one level of categorical variable

I have a model with the following parameters: Groups: factor - 4 levels (base level = control group) Time: numerical Label = factor - 3 levels (base level = control group) price = numerical (5 ...
  • 11
3 votes
0 answers
184 views

High concurvity/ collinearity between time and temperature in GAM predicting deaths but low ACF. Does this matter?

I have problems with concurvity in my thesis research, and don't have access to a statistician who works with time series, GAMs or R. I'd be very grateful for any (partial) answer, however short. I'll ...
  • 139
0 votes
0 answers
64 views

Should I handle multicollinearity before or after my feature selection?

I'm conducting regression analysis and I'm wondering if I should handle multicollinearity before or after my chosen feature selection method. The data I am using produces VIF values greater than 10 ...
0 votes
0 answers
32 views

Can two predictor variables be collinear and independent/dependent at the same time?

I have these variables: y1 - dichotomous 0/1 , as response y2 - continuous , as response x1 - dichotomous 0/1 , as predictor x2 - dichotomous 0/1 , as predictor x3 - continuous , as predictor ... and ...
0 votes
0 answers
57 views

Is collinearity really not a problem for GLM?

I have read that collinearity is not a problem for GLM. Is it really? I here estimate two models. The dependent variable is default, a dummy equals to 1 if someone ...
1 vote
0 answers
22 views

How to Assess Multicollinearity for Independent Count and Rate Variables

How do I test an independent variable for multicollinearity if it comes from a Poisson or Negative Binomial distribution? A common approach for testing the multicollinearity of a model's independent ...
  • 227
0 votes
1 answer
112 views

How does perfect multicollinearity affect $R^2$ and $R_{\text{adj}}^2$?

I'd like to know how does perfect collinearity affect measures of fit (R squared and R squared adjusted). A mathematical approach is not necessary, just the general intuition is fine.
1 vote
0 answers
37 views

Is there a nonlinear counterpart of multicollinearity? [closed]

Is there a nonlinear counterpart of multicolinearity. i.e. if two variables x1 and x2 are nonlinearly dependent on eachother in such a way that makes interpretation the coefficients of a specific ...
0 votes
0 answers
80 views

Multicollinearity and factor analysis

I’m carrying out and principal axis factor analysis to validate an existing questionnaire in a different population. The determinant of the R-matrix is incredibly low and way below the threshold 0....
  • 1
1 vote
2 answers
47 views

How to measure marginal effect of interdependent variables on a binary outcome?

My data represents observations on a possible sequences of events that may lead to a positive outcome (y). Each event (A, B, C, D) is dependent on the previous event; for D to occur C must occur, for ...
0 votes
0 answers
46 views

Multicollinearity in polynomial regression in R

I fit a model to my data with the following formula after a stepwise selection (x1 to x5 stand for my variables): lm(formula = outcome ~ x1+ x3 + x4 + x2+ x5+ poly(x4, 2) + poly(x2, 2) + poly(x5, 2) + ...
  • 1
0 votes
0 answers
42 views

GMM clustering with binary and multicollinear data

I am using GMM clustering on bank data. The data have both categorical and numerical attributes. The categorical data were converted to numerical using binary encoding. I have a couple of questions: ...
0 votes
0 answers
48 views

interaction between (possible) correlated variables

Suppose I have a set of variables a,b,c,d,f,g,h,i, and I created a composite score by summing these eight variables into a composite score e=(a+b+c+d+f+g+h+i). Now I want to explore the interaction on ...
4 votes
0 answers
67 views

PCA versus mixed effects model: Incorporating relationship between loadings?

I have species abundances with associations between environmental variables. I realize the RDA will only be able to tell you the strength of the relationship of all the species abundances with the ...
1 vote
1 answer
62 views

Multicolinearity ONLY raises the variance of the coefficient estimates?

A hypothetical conversation: Person A: I am building a forecasting model. It is a logistic regression. All coefficients are statistically significant at the 5% level. I calculated the VIF for the set ...
0 votes
0 answers
22 views

Assessing combined effects of binary predictor variables

I'm looking to analyse the effect of 20-30 binary predictor variables on a continuous response variable. I'm no statistician, so first off I'm not sure whether a regression with this many binary ...
0 votes
0 answers
31 views

How to adjust regression for colinearity within certain levels of categorical variable?

I'm analyzing results from a large workforce survey, with approximately 110,000 responses to each question. We are interested in how certain demographic variables impact employees' opinions, so we ...
0 votes
1 answer
65 views

Interpreting Logistic Regression Coefficients Under Collinearity

I thought this would be an easy question to find an answer to, but for the life of me I am having trouble finding anything that fully addresses my current problem: Consider a situation where we are ...
0 votes
0 answers
55 views

Multicollinearity: not by some measures (VIF, TOL), yes by others (Wi, Fi) - does this invalidate the regression model?

Data is from http://peopleanalytics-regression-book.org/data/sociological_data.csv After reading this into R as sociological_data, here is my code: ...
0 votes
1 answer
150 views

Multicollinearity iff determinant correlation matrix = 0

I'm studying Linear Models again, after finishing my degree some years ago. I found in my old notes that, according to my professor, one can check multicollinearity calculating the determinant of the ...
0 votes
1 answer
92 views

Why do I find that OLS linear regression is robust against colinearity?

As per the textbook, OLS should fail when using colinear covariates. On their LinearRegression() documentation, sklearn states: When features are correlated and the columns of the design matrix have ...
  • 105
1 vote
0 answers
82 views

Is the non-multicollinearity assumption for OLS multiple regression just an assumption of convenience?

The four assumptions for bivariate regression are:     • (L)inearity     • (I)ndepdent observations     • (N)ormal errors     • (E)qual variance And for multiple regression we add a fifth assumption:  ...
  • 3,933
0 votes
0 answers
24 views

Coefficients interpretation after recovering raw coefficients from a regression which used orthogonal regressors

I have an unbalanced panel of 747 observations and 15 years. After testing for Pooled, FE and RE, FE is the "best" model. However, I have multicollinearity problems. I can either remove one ...
1 vote
1 answer
128 views

If a regression problem is ill-conditioned, does that mean we cannot perform SGD? What happens if we do?

By ill-conditioned regression problem, I mean that the feature matrix $X$ is not full rank. For example, X contains two or more columns highly correlated. If that's the case, $X^T\cdot X$ is not ...
  • 185

1
2 3 4 5
23