# Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

1,220 questions
Filter by
Sorted by
Tagged with
1 vote
12 views

### Estimating correlated features in OLS regression [closed]

I am given decently large sample set of $y$’s and ${\beta}_{i}$’s of following OLS fitted by someone else: $y$ = $\sum({\beta}_{i} x_i)$ + α + ε I want to estimate $x_i$’s (say $\hat{x_i}$’s). ...
• 235
1 vote
21 views

### ridge regression vs enforcing coefficient signs when regularizing colinearity

When dealing with multicollinearity problem, one standard solution is to run ridge regression. So we avoid a fit of $y = 10000 x_1 - 9999 x_2$ for example. when $x_1$ and $x_2$ are highly correlated....
• 230
51 views

### Does including a variable for stepwise change and another variable for linear change in an ARIMAX model introduce issues of multicollinearity?

I am trying to perform interrupted time series analysis using an ARIMA model based on the following paper: Interrupted time series analysis using autoregressive integrated moving average (ARIMA) ...
50 views

### Multicollinearity when controlling for a variable

I have a few questions about multicollinearity in my data: I'm looking at a certain type of lesion seen on MRI scans; for each patient I know the volume of those lesions and a metric that captures the ...
113 views

### Difference-in-Differences with Multiple Time Periods

For my master thesis, I want to estimate the effect that replications have on the citations of a paper. For this, I wanted to make a comparison between the citations of papers that were once ...
1 vote
32 views

### GLMM Model Averaging with Predictor Multicollinearity

I am running GLMM models to determine how environmental factors influence bird collisions. I've obtained a list of candidate models with delta AIC less than 2, and I want to perform model averaging. I ...
• 11
1 vote
23 views

### how to find the singular values of singular value decomposition (SVD) [closed]

I am a bit confused as to how we find the singular values and therefore condition index number. Some mathematicians say the singular values are the square roots of the eigenvalues of the correlation ...
• 23
1 vote
80 views

### No Multicollinearity between highly Correlated IVs

Suppose I'm building a multiple regression model, with y as dependent variable and X1, X2,..., Xn as independent variables. I've ...
• 6,416
104 views

### Any Insights on the adoption and use of the Healthy Akaike Information Criterion (hAIC)?

Recently, I came across the Healthy Akaike Information Criterion (hAIC), introduced by Demidenko in his 2004 book "Mixed Models: Theory and Applications with R." Despite its (potential) ...
• 63.6k
182 views

### Test for multicollinearity with binary and continuous independent variables

I have a question concerning multicollinearity: I have several independent variables. Some are binary and some continuous. The dependent variable is binary. Can I use the Pearson correlations to test ...
• 31
49 views

### How to address multicollinearity when adding random effects in gam model and include random slopes?

I'm working with a generalized additive model (GAM) using the mgcv package in R. My dataset includes measurements collected over several years at different sites. I ...
• 367
1 vote
44 views

### Understanding the coefficients of highly correlated features in generalized linear models

I am trying to fit a generalized linear model, for simplicity assume that is a linear regression. I have a bunch of features and I fitted a linear model to it, the feature ...
• 13
61 views

• 2,311
1 vote
25 views

### Dealing with collinearity [closed]

I have a variable called Instagram reach, which represents the number of people who viewed a post, and engagement is the number of unique individuals who interacted with that post. We know that ...
31 views

### How to interpret the coefficient of a residualized variable in a linear model?

I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
42 views

### Multicollinearity with interaction term and fixed effects

I have the following regression including an interaction term and fixed effects ...
21 views

### Is the random forest classfier affected by related samples or biological replicates?

Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
• 11
25 views

### How to handle multicollinearity with varying length and type of conjoint treatments?

We ran a complicated experiment and are struggling to build a linear model that estimates everything we're interested in. We showed each person a description of a product (for illustration, let's say ...
• 51
301 views

### Multicollinearity and control variables dilemma

I had some superficial understanding of multicollinearity, that two highly correlated variables in the regression model are not what we want, as the estimated coefficient would be biased. Control ...
• 331
75 views

### Multivariate statistical criterion to select key variables

I have a complex dataset with several predictor variables $X_i$ ($i=1,...,m$) and several outcome variables $Y_j$ $(j=1,...,n$). The problem is that many of the predictor variables are correlated ...
48 views

### Distribution of Maximum Eigen Value

Suppose I have X, k*n, where $M=X'X$. Suppose $n>>k$, and $rank(M) =k-1$. Suppose $\lambda_1, \cdots, \lambda_{k-1}$ are the eigen values of M. Under the assumption that the columns of X are ...
• 265
345 views

### How to report the interaction effect in the paper?

I am calculating the moderation (interaction) effect of one variable on the relationship between two variables using Mplus. All of them are continuous variables. Before I calculated the interaction ...
• 141
48 views

### Choosing Predictors in Multiple Regression

I am planning regression analyses and present this (hypothetical) scenario to communicate my query. I am interested in the effect of 2 different measures ('IQ' and 'SPQ') on dependent variable '...
• 512
1 vote
44 views

### To avoid the problem of perfect multicollinearity in binary variables, why is it fine to create separate variables for them in the linear regression? [duplicate]

I am reading Wooldridge's econometrics textbook. He provides the following example: Suppose we have a regression model relating wages to gender and education level. wage = b0 + d0 female + b1educ + u ...
• 309
87 views

### Fixed-effect model with ridge regression, or how else to deal with multicollinearity

I am currently writing a registered report for data which will be clustered within eight countries. Since that is too few to do a multilevel model with random effects (McNeish & Stapleton, 2016), ...
1 vote
47 views

### Mediation analysis not producing expected results, probably as consequence of multicollinearity issue

I am using the PROCESS macro to do a basic mediation analysis, with variables X,M, and Y. All are continuous. I know from theory that X and M, and M and Y respectively, should have a relationship. I ...
• 111
59 views

### OLS Duplicated Feature

Let's say we have $n$ samples and a single feature variable $X$, and we run OLS to regress $Y$ onto $X$ and get some $\beta$. Now, suppose we duplicate this feature to now get $X_1=X_2$, and we ...
93 views

### Losing significance when adding variables to hierarchical regression model

I have two hierarchical models with continuous variables. In the first block, one of the variables is significant. However, in the second block, when I add three more variables the first variable ...
124 views

### multicollinearity and categorical variables

When performing regression with categorical variables, in order to avoid multicollinearity, it is necessary to drop one level. This is clear in fact: Let's assume I have a binary categorical variable (...
1 vote
70 views

### Issue of multicollinearity in R for glm analysis

I was wondering if someone could help me with a statistical problem I have run into. Any help would be incredibly helpful. Please note that for clarity, I have simplified the below description. It ...
582 views

### Does it make sense to talk of "multicollinearity" in the context of simple linear regression?

As far as I am concerned, "multicollinearity" referers to the presence of collinearity between two or more variables, even if there is no pair of variables that have a particularly high ...
1 vote
167 views

### Homoskedasticity and Collinearity

I am curious whether the property of homoskedasticity is more or less dependent on the correlation between independent variables. I assumed that if the $cor(x_1,x_2....x_n) \approx 0$, hence the ...
65 views

### Assumption in DBSCAN Clustering

Does the non-multicollinearity assumption apply to DBSCAN? I've read that this clustering method makes no assumptions about the density or variance in clusters that may exist in the data set. Can that ...
• 3
50 views

### How to deal with interaction terms in regression that cannot have a negative product?

Assume we have the following model: $y = \beta_0 + \alpha_1 * x_1 ^{\beta_1} + \alpha_2 * x_2^{\beta_2} + \alpha_3 * x_1^{\beta_1} * x_2^{\beta_2}$ where as we have the following priors for our IV's \$\...
1 vote
187 views

### Could multicollinearity be messing up my logistic regression? Can I overcome it?

My data has 5 binary dependent variables, 9 categorical independent variables, and 3 continuous independent variables, with a sample size of 1232. The 5 dependent variables are just different ways of ...
• 113