Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
2 votes
2 answers
38 views

Does Factor Analysis completely mitigate the singular covariance matrix problem?

Background I have been trying to understand Stanford CS 229’s lecture about Factor Analysis and the accompanying lecture notes. The lecturer introduced Factor Analysis as a way to mitigate the ...
fumoboy007's user avatar
0 votes
0 answers
12 views

Check_model performance package interpretation plots

I am fairly new to r and statistic and I am building GLMs for frogs occupancy and abundance using a dataset with 57 observations and 13 independent variables. As some variables are correlated the ...
Marco Lassandro's user avatar
0 votes
0 answers
12 views

Estimating treatment effect with/without intercept [duplicate]

I am trying to estimate the treatment effect based on the above two model: $$Y(Z)=\beta_0+\tau Z+\varepsilon.$$ $$Y(Z)=\tau Z+\varepsilon.$$ Based on result from my data, I found the intercept is not ...
Fangzhi Luo's user avatar
0 votes
3 answers
61 views

searching for which x-variables affect y the most when there is strong collinearity among the x-variables [closed]

I have a sample of 33 observations with 16 variables. My main goal is to find which (could be several) of these affect y the most. Y is an unwanted error in a system and the x variables are possible ...
anders's user avatar
  • 101
3 votes
1 answer
85 views

Error in mixed-models. Which to detect? Collinearity? Singularity in backsolve at level 0, block 1

Firstly, I would like to admit that even though it is not the first time I am working with linear mixed models, the mathematical foundations escape me. I am running a linear mixed-effects model using ...
Javier Hernando's user avatar
0 votes
0 answers
49 views

Fixed effects regression drops estimates due to collinearity for more aggregated FE, but not for less aggregated FE

I am running a fixed effects regression in r using the fixest package with a few different settings. I cannot provide data that replicates the errors, as the ...
flâneur's user avatar
  • 133
0 votes
0 answers
9 views

Which variables should I include/exclude in my regression analysis for GDP? [duplicate]

I am wanting to establish the impact mobile money has on Kenyan economic growth. I am building my regression model and have collected data for GDP, consumption, government spending, investment, net ...
Georgia's user avatar
1 vote
1 answer
29 views

How this categorical variable have no reference value?

I'm reading the vignette for fixest After fitting this model for trade between countries: $E\left(Trade_{i,j,p,t}\right)=\exp\left(\gamma_{i}^{Exporter}+\gamma_{j}^{Importer}+\gamma_{p}^{Product}+\...
robertspierre's user avatar
1 vote
0 answers
22 views

Dealing with collinearity [closed]

I have a variable called Instagram reach, which represents the number of people who viewed a post, and engagement is the number of unique individuals who interacted with that post. We know that ...
Thiago Cunha's user avatar
0 votes
0 answers
11 views

How to interpret the coefficient of a residualized variable in a linear model?

I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
Peter's user avatar
  • 1
0 votes
0 answers
25 views

Multicollinearity with interaction term and fixed effects

I have the following regression including an interaction term and fixed effects ...
user avatar
0 votes
0 answers
16 views

Is the random forest classfier affected by related samples or biological replicates?

Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
Tal's user avatar
  • 11
0 votes
0 answers
22 views

How to handle multicollinearity with varying length and type of conjoint treatments?

We ran a complicated experiment and are struggling to build a linear model that estimates everything we're interested in. We showed each person a description of a product (for illustration, let's say ...
Tia's user avatar
  • 51
3 votes
1 answer
242 views

Multicollinearity and control variables dilemma

I had some superficial understanding of multicollinearity, that two highly correlated variables in the regression model are not what we want, as the estimated coefficient would be biased. Control ...
LJNG's user avatar
  • 331
0 votes
1 answer
75 views

Multivariate statistical criterion to select key variables

I have a complex dataset with several predictor variables $X_i$ ($i=1,...,m$) and several outcome variables $Y_j$ $(j=1,...,n$). The problem is that many of the predictor variables are correlated ...
Fernando's user avatar
2 votes
1 answer
47 views

Distribution of Maximum Eigen Value

Suppose I have X, k*n, where $M=X'X$. Suppose $n>>k$, and $rank(M) =k-1$. Suppose $\lambda_1, \cdots, \lambda_{k-1}$ are the eigen values of M. Under the assumption that the columns of X are ...
deb's user avatar
  • 265
6 votes
2 answers
185 views

How to report the interaction effect in the paper?

I am calculating the moderation (interaction) effect of one variable on the relationship between two variables using Mplus. All of them are continuous variables. Before I calculated the interaction ...
İpek Gülsün's user avatar
2 votes
1 answer
46 views

Choosing Predictors in Multiple Regression

I am planning regression analyses and present this (hypothetical) scenario to communicate my query. I am interested in the effect of 2 different measures ('IQ' and 'SPQ') on dependent variable '...
SilvaC's user avatar
  • 512
1 vote
0 answers
44 views

To avoid the problem of perfect multicollinearity in binary variables, why is it fine to create separate variables for them in the linear regression? [duplicate]

I am reading Wooldridge's econometrics textbook. He provides the following example: Suppose we have a regression model relating wages to gender and education level. wage = b0 + d0 female + b1educ + u ...
user57623's user avatar
  • 309
0 votes
1 answer
70 views

Fixed-effect model with ridge regression, or how else to deal with multicollinearity

I am currently writing a registered report for data which will be clustered within eight countries. Since that is too few to do a multilevel model with random effects (McNeish & Stapleton, 2016), ...
Maximilian's user avatar
1 vote
1 answer
35 views

Mediation analysis not producing expected results, probably as consequence of multicollinearity issue

I am using the PROCESS macro to do a basic mediation analysis, with variables X,M, and Y. All are continuous. I know from theory that X and M, and M and Y respectively, should have a relationship. I ...
Maarten 's user avatar
0 votes
0 answers
47 views

OLS Duplicated Feature

Let's say we have $n$ samples and a single feature variable $X$, and we run OLS to regress $Y$ onto $X$ and get some $\beta$. Now, suppose we duplicate this feature to now get $X_1=X_2$, and we ...
PerplexedPelican's user avatar
0 votes
1 answer
46 views

Losing significance when adding variables to hierarchical regression model

I have two hierarchical models with continuous variables. In the first block, one of the variables is significant. However, in the second block, when I add three more variables the first variable ...
Statistics_3280's user avatar
2 votes
1 answer
94 views

multicollinearity and categorical variables

When performing regression with categorical variables, in order to avoid multicollinearity, it is necessary to drop one level. This is clear in fact: Let's assume I have a binary categorical variable (...
Marco De Virgilis's user avatar
1 vote
0 answers
59 views

Issue of multicollinearity in R for glm analysis

I was wondering if someone could help me with a statistical problem I have run into. Any help would be incredibly helpful. Please note that for clarity, I have simplified the below description. It ...
Ian Holdroyd's user avatar
3 votes
3 answers
546 views

Does it make sense to talk of "multicollinearity" in the context of simple linear regression?

As far as I am concerned, "multicollinearity" referers to the presence of collinearity between two or more variables, even if there is no pair of variables that have a particularly high ...
ghost wizard's user avatar
1 vote
1 answer
156 views

Homoskedasticity and Collinearity

I am curious whether the property of homoskedasticity is more or less dependent on the correlation between independent variables. I assumed that if the $cor(x_1,x_2....x_n) \approx 0$, hence the ...
Tunay Sabri Yüksel's user avatar
0 votes
0 answers
49 views

Assumption in DBSCAN Clustering

Does the non-multicollinearity assumption apply to DBSCAN? I've read that this clustering method makes no assumptions about the density or variance in clusters that may exist in the data set. Can that ...
Anna's user avatar
  • 3
0 votes
1 answer
47 views

How to deal with interaction terms in regression that cannot have a negative product?

Assume we have the following model: $y = \beta_0 + \alpha_1 * x_1 ^{\beta_1} + \alpha_2 * x_2^{\beta_2} + \alpha_3 * x_1^{\beta_1} * x_2^{\beta_2}$ where as we have the following priors for our IV's $\...
richard baws's user avatar
1 vote
1 answer
64 views

Could multicollinearity be messing up my logistic regression? Can I overcome it?

My data has 5 binary dependent variables, 9 categorical independent variables, and 3 continuous independent variables, with a sample size of 1232. The 5 dependent variables are just different ways of ...
Jamie Watts's user avatar
3 votes
1 answer
90 views

high variance proportion for intercept

I'm using the ols_eigen_cindex function to assess multicollinearity. With these variance proportions: ...
locus's user avatar
  • 1,603
0 votes
1 answer
124 views

Fixed Effects causes Multicollinearity

I have a question regarding my regression model and would be grateful for any help. I have a dataset that contains every tranche of a Deal (a Deal may contain multiple Tranches) and its Variables like ...
Nasim El-Issa's user avatar
5 votes
1 answer
127 views

Should I orthogonalize variables before regression?

If I have several correlated variables in my dataset which I would like to include as predictors in my model. For example with this simulated dataset: ...
locus's user avatar
  • 1,603
0 votes
0 answers
32 views

Is using VIF to Select Lambda in Ridge Regression a valid approach?

I recently came across an article that suggests selecting the lambda parameter in ridge regression based on Variance Inflation Factor (VIF) values. The method aims to choose a lambda that ensures all ...
HiddenLeafCoder's user avatar
3 votes
2 answers
397 views

Addressing Multicollinearity

Say you regress $Y = x_1,….x_k$ and find out that you have multicollinearity. I propose the following solution: Say you suspect $x_1$ and $x_2$ are collinear. I regress $x_1$ on $x_2$ and get the ...
ayedeeyay's user avatar
2 votes
1 answer
37 views

Feature selection in a traditional regression model to an experiment data

I have an experiment data (total of 96) with 10 predictor and 2 response variables. I want to build a traditional multiple linear regression model to them in R. My aim is to build clearly ...
egeku's user avatar
  • 21
1 vote
1 answer
90 views

Can multicollinearity of parametric terms in a GAM be ignored?

I've had such a hard time finding a source that addresses how to handle multicollinearity of parametric terms in a generalized additive model (GAM) that I'm starting to think it may not be important, ...
Nate's user avatar
  • 1,529
2 votes
2 answers
317 views

What's the difference between a beta coef. estimate from linear regression and a VIF value?

I'm having trouble deciding what to do when two important variables (important to my research questions) are seemingly too related, but both need to be included in a full model (this is a longitudinal ...
Nate's user avatar
  • 1,529
0 votes
0 answers
66 views

How to deal with insignificance in this regression?

The hw question: Data set includes: ...
Dannis's user avatar
  • 23
0 votes
0 answers
40 views

How to deal with insignificance and multicolinearity? [closed]

When performing a regression analysis to test whether Asians have the lowest COVID mortality rates using stata, none of variables seem to have significance other than population. How do I determine ...
Dannis's user avatar
  • 23
5 votes
2 answers
401 views

OLS assumption, full rank of matrix $X$

One of the OLS assumptions concerning the $X$-matrix (with a constant) is that the columns $(1, x_{i1}, \ldots , x_{iK})$ are not linearly dependent. This looks intuitive to me, because of the dummy-...
Marlon Brando's user avatar
1 vote
1 answer
90 views

How to Combine Correlated Binary Variables

I have a count data, and most of my explanatory variables are binary (they represent different types of support given to a group). Three variables seem to be correlated with each other. I don't want ...
Nickie's user avatar
  • 11
0 votes
0 answers
27 views

Does it make sense to define non-multicollinearity as "an assumption" that can be "tested"?

In the case of multicollinearity, I wonder why: We typically talk about lack of it as an assumption (thus, we assume non-multicollinearity): https://www.statology.org/multiple-linear-regression-...
Federico Tedeschi's user avatar
1 vote
0 answers
54 views

Two-Way Fixed Effects Model in R - Error - Multicollinearity

I am trying to build a two way fixed effects model (entity-specific and time-specific fixed effects). However, when I run the fixed effects code in R, I get the following error message: ...
carol's user avatar
  • 11
2 votes
3 answers
490 views

Can I keep several features with multicollinearity in the model?

I am creating a sports prediction model with around 300 features. There is a high degree of multicollinearity which breaks an assumption for logistic regression. The issue is that several of these ...
sla813's user avatar
  • 55
3 votes
3 answers
384 views

An independent variable that is correlated with another variable in a regression model

I'm doing a regression analysis to understand the relationship between disease severity (Severity) and viral load (VL). The <...
Michael's user avatar
  • 33
0 votes
0 answers
19 views

Time-series lags and mulicollinearity

In the context of regression analysis, why time-series analysts do not seem to be bother (I think) by multicollinearity when using AR terms or predictors lags. The main objective is inference rather ...
Nip's user avatar
  • 561
1 vote
0 answers
28 views

How do deal with multicollinearity, endogeneity and interpret the interaction terms in a panel dataset?

The model ŷ = b0 + b1X1 + b2X2 + b3X1X2 ŷ =company financial performance metric X1 = carbon emissions X2 = carbon assurance X1X2 = interaction term The issues: Let’s say: • X1 + X2 are related (but ...
Reuben's user avatar
  • 11
1 vote
2 answers
342 views

Multicollinearity in Multiple Regression with SPSS

I want to run a multiple regression in SPSS with 7 independent variables but 3 of them are showing high correlation coefficients in the correlation matrix. How do I diagnose multicollinearity?
Rati's user avatar
  • 21
0 votes
0 answers
21 views

About centering to accommodate multicollinearity (Ordinal Logistic Regression Analysis, Logistic Regression Analysis)

When interaction terms are used in multiple regression analysis, often centralization of the variables (subtracting the mean of the variable from each variable) is used to deal with multicollinearity, ...
946's user avatar
  • 1

1
2 3 4 5
25