Questions tagged [multicollinearity]
Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.
1,220
questions
1
vote
0
answers
12
views
Estimating correlated features in OLS regression [closed]
I am given decently large sample set of $y$’s and ${\beta}_{i}$’s of following OLS fitted by someone else:
$y$ = $\sum({\beta}_{i} x_i)$ + α + ε
I want to estimate $x_i$’s (say $\hat{x_i}$’s). ...
- 235
1
vote
0
answers
21
views
ridge regression vs enforcing coefficient signs when regularizing colinearity
When dealing with multicollinearity problem, one standard solution is to run ridge regression. So we avoid a fit of $y = 10000 x_1 - 9999 x_2 $ for example. when $x_1$ and $x_2$ are highly correlated....
- 230
0
votes
1
answer
51
views
Does including a variable for stepwise change and another variable for linear change in an ARIMAX model introduce issues of multicollinearity?
I am trying to perform interrupted time series analysis using an ARIMA model based on the following paper: Interrupted time series analysis using autoregressive integrated moving average (ARIMA) ...
2
votes
2
answers
50
views
Multicollinearity when controlling for a variable
I have a few questions about multicollinearity in my data: I'm looking at a certain type of lesion seen on MRI scans; for each patient I know the volume of those lesions and a metric that captures the ...
- 63
2
votes
1
answer
113
views
Difference-in-Differences with Multiple Time Periods
For my master thesis, I want to estimate the effect that replications have on the citations of a paper. For this, I wanted to make a comparison between the citations of papers that were once ...
1
vote
1
answer
32
views
GLMM Model Averaging with Predictor Multicollinearity
I am running GLMM models to determine how environmental factors influence bird collisions. I've obtained a list of candidate models with delta AIC less than 2, and I want to perform model averaging.
I ...
- 11
1
vote
0
answers
23
views
how to find the singular values of singular value decomposition (SVD) [closed]
I am a bit confused as to how we find the singular values and therefore condition index number. Some mathematicians say the singular values are the square roots of the eigenvalues of the correlation ...
- 23
1
vote
1
answer
80
views
No Multicollinearity between highly Correlated IVs
Suppose I'm building a multiple regression model, with y as dependent variable and X1, X2,..., Xn as independent variables. I've ...
- 6,416
9
votes
0
answers
104
views
Any Insights on the adoption and use of the Healthy Akaike Information Criterion (hAIC)?
Recently, I came across the Healthy Akaike Information Criterion (hAIC), introduced by Demidenko in his 2004 book "Mixed Models: Theory and Applications with R." Despite its (potential) ...
- 63.6k
3
votes
2
answers
182
views
Test for multicollinearity with binary and continuous independent variables
I have a question concerning multicollinearity: I have several independent variables. Some are binary and some continuous. The dependent variable is binary. Can I use the Pearson correlations to test ...
- 31
2
votes
1
answer
49
views
How to address multicollinearity when adding random effects in gam model and include random slopes?
I'm working with a generalized additive model (GAM) using the mgcv package in R. My dataset includes measurements collected over several years at different sites. I ...
- 367
1
vote
1
answer
44
views
Understanding the coefficients of highly correlated features in generalized linear models
I am trying to fit a generalized linear model, for simplicity assume that is a linear regression. I have a bunch of features and I fitted a linear model to it, the feature ...
- 13
4
votes
2
answers
61
views
Is lasso preferable to ridge or principal component regression in multicollinear settings?
Consider a $N\times p$ data matrix $\mathbf X$ with columns $\mathbf x_j$. ESL recommends standardizing the inputs before performing ridge regression, which I understand to mean centering the columns $...
- 140
0
votes
0
answers
13
views
Connection between multicollinearity and problem of identification in Simultaneous Equations Model
Is there any connection between multicollinearity and problem of identification in Simultaneous Equations Model?
I know Multicollinearity is the occurrence of high intercorrelations among two or more ...
1
vote
1
answer
38
views
What is the best model for this case?
I have the following problem:
A data set, which is about the soft drink consumption of people, that
covers 300 subjects are available to us. Using Excel tabulations and
graphing capabilities only:
...
- 103
2
votes
0
answers
41
views
Regression with small sample size - LASSO or remove variables?
I'm trying to run a regression, but I only have 14 observations, each being a different city in the US. My dependent variable is the total number of trips per capita, and my explanatory variables are ...
2
votes
0
answers
30
views
Is this multicollinearity, and how can I specify my model better?
I'm analyzing data from the usual care period only of a stepped wedge cluster-randomized trial. The goal is to describe the usual care period as though it was a cohort study because much higher ...
- 43
0
votes
1
answer
45
views
Understanding differences in collinearity across Stata commands
Here's a simple example of a regression of y on x including time and id fixed-effects and both a linear and a quadratic time trend. If my t starts at 1, these 3 different regressions get the same ...
- 15
3
votes
2
answers
50
views
Does Factor Analysis completely mitigate the singular covariance matrix problem?
Background
I have been trying to understand Stanford CS 229’s lecture about Factor Analysis and the accompanying lecture notes. The lecturer introduced Factor Analysis as a way to mitigate the ...
- 131
0
votes
0
answers
85
views
Check_model performance package interpretation plots
I am fairly new to R and statistics and I am building GLMs for frogs occupancy and abundance using a dataset with 57 observations and 13 independent variables. As some variables are correlated the ...
0
votes
0
answers
12
views
Estimating treatment effect with/without intercept [duplicate]
I am trying to estimate the treatment effect based on the above two model:
$$Y(Z)=\beta_0+\tau Z+\varepsilon.$$
$$Y(Z)=\tau Z+\varepsilon.$$
Based on result from my data, I found the intercept is not ...
- 121
2
votes
3
answers
78
views
searching for which x-variables affect y the most when there is strong collinearity among the x-variables [closed]
I have a sample of 33 observations with 16 variables. My main goal is to find which (could be several) of these affect y the most. Y is an unwanted error in a system and the x variables are possible ...
- 121
3
votes
1
answer
109
views
Error in mixed-models. Which to detect? Collinearity? Singularity in backsolve at level 0, block 1
Firstly, I would like to admit that even though it is not the first time I am working with linear mixed models, the mathematical foundations escape me. I am running a linear mixed-effects model using ...
- 275
0
votes
0
answers
84
views
Fixed effects regression drops estimates due to collinearity for more aggregated FE, but not for less aggregated FE
I am running a fixed effects regression in r using the fixest package with a few different settings. I cannot provide data that replicates the errors, as the ...
- 65
0
votes
0
answers
9
views
Which variables should I include/exclude in my regression analysis for GDP? [duplicate]
I am wanting to establish the impact mobile money has on Kenyan economic growth. I am building my regression model and have collected data for GDP, consumption, government spending, investment, net ...
- 1
1
vote
1
answer
34
views
How this categorical variable have no reference value?
I'm reading the vignette for fixest
After fitting this model for trade between countries:
$E\left(Trade_{i,j,p,t}\right)=\exp\left(\gamma_{i}^{Exporter}+\gamma_{j}^{Importer}+\gamma_{p}^{Product}+\...
- 2,311
1
vote
0
answers
25
views
Dealing with collinearity [closed]
I have a variable called Instagram reach, which represents the number of people who viewed a post, and engagement is the number of unique individuals who interacted with that post. We know that ...
- 11
0
votes
0
answers
31
views
How to interpret the coefficient of a residualized variable in a linear model?
I was fitting a linear model and there was strong multicollinearity present in the data. So, I decided to residualize one regressor variable to reduce the multicollinearity and fitted the model again. ...
- 1
0
votes
0
answers
42
views
Multicollinearity with interaction term and fixed effects
I have the following regression including an interaction term and fixed effects
...
anrisakaki96
0
votes
0
answers
21
views
Is the random forest classfier affected by related samples or biological replicates?
Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
- 11
0
votes
0
answers
25
views
How to handle multicollinearity with varying length and type of conjoint treatments?
We ran a complicated experiment and are struggling to build a linear model that estimates everything we're interested in. We showed each person a description of a product (for illustration, let's say ...
- 51
3
votes
1
answer
301
views
Multicollinearity and control variables dilemma
I had some superficial understanding of multicollinearity, that two highly correlated variables in the regression model are not what we want, as the estimated coefficient would be biased.
Control ...
- 331
0
votes
1
answer
75
views
Multivariate statistical criterion to select key variables
I have a complex dataset with several predictor variables $X_i$ ($i=1,...,m$) and several outcome variables $Y_j$ $(j=1,...,n$). The problem is that many of the predictor variables are correlated ...
- 3
2
votes
1
answer
48
views
Distribution of Maximum Eigen Value
Suppose I have X, k*n, where $M=X'X$. Suppose $n>>k$, and $rank(M) =k-1$. Suppose $\lambda_1, \cdots, \lambda_{k-1}$ are the eigen values of M. Under the assumption that the columns of X are ...
- 265
6
votes
2
answers
345
views
How to report the interaction effect in the paper?
I am calculating the moderation (interaction) effect of one variable on the relationship between two variables using Mplus. All of them are continuous variables. Before I calculated the interaction ...
- 141
2
votes
1
answer
48
views
Choosing Predictors in Multiple Regression
I am planning regression analyses and present this (hypothetical) scenario to communicate my query.
I am interested in the effect of 2 different measures ('IQ' and 'SPQ') on dependent variable '...
- 512
1
vote
0
answers
44
views
To avoid the problem of perfect multicollinearity in binary variables, why is it fine to create separate variables for them in the linear regression? [duplicate]
I am reading Wooldridge's econometrics textbook. He provides the following example: Suppose we have a regression model relating wages to gender and education level.
wage = b0 + d0 female + b1educ + u
...
- 309
0
votes
1
answer
87
views
Fixed-effect model with ridge regression, or how else to deal with multicollinearity
I am currently writing a registered report for data which will be clustered within eight countries. Since that is too few to do a multilevel model with random effects (McNeish & Stapleton, 2016), ...
1
vote
1
answer
47
views
Mediation analysis not producing expected results, probably as consequence of multicollinearity issue
I am using the PROCESS macro to do a basic mediation analysis, with variables X,M, and Y. All are continuous. I know from theory that X and M, and M and Y respectively, should have a relationship. I ...
- 111
0
votes
0
answers
59
views
OLS Duplicated Feature
Let's say we have $n$ samples and a single feature variable $X$, and we run OLS to regress $Y$ onto $X$ and get some $\beta$. Now, suppose we duplicate this feature to now get $X_1=X_2$, and we ...
0
votes
1
answer
93
views
Losing significance when adding variables to hierarchical regression model
I have two hierarchical models with continuous variables. In the first block, one of the variables is significant. However, in the second block, when I add three more variables the first variable ...
2
votes
1
answer
124
views
multicollinearity and categorical variables
When performing regression with categorical variables, in order to avoid multicollinearity, it is necessary to drop one level. This is clear in fact:
Let's assume I have a binary categorical variable (...
1
vote
0
answers
70
views
Issue of multicollinearity in R for glm analysis
I was wondering if someone could help me with a statistical problem I have run into. Any help would be incredibly helpful.
Please note that for clarity, I have simplified the below description. It ...
- 63
3
votes
3
answers
582
views
Does it make sense to talk of "multicollinearity" in the context of simple linear regression?
As far as I am concerned, "multicollinearity" referers to the presence of collinearity between two or more variables, even if there is no pair of variables that have a particularly high ...
- 91
1
vote
1
answer
167
views
Homoskedasticity and Collinearity
I am curious whether the property of homoskedasticity is more or less dependent on the correlation between independent variables. I assumed that if the $cor(x_1,x_2....x_n) \approx 0$, hence the ...
0
votes
0
answers
65
views
Assumption in DBSCAN Clustering
Does the non-multicollinearity assumption apply to DBSCAN?
I've read that this clustering method makes no assumptions about the density or variance in clusters that may exist in the data set. Can that ...
- 3
0
votes
1
answer
50
views
How to deal with interaction terms in regression that cannot have a negative product?
Assume we have the following model: $y = \beta_0 + \alpha_1 * x_1 ^{\beta_1} + \alpha_2 * x_2^{\beta_2} + \alpha_3 * x_1^{\beta_1} * x_2^{\beta_2}$ where as we have the following priors for our IV's $\...
1
vote
1
answer
187
views
Could multicollinearity be messing up my logistic regression? Can I overcome it?
My data has 5 binary dependent variables, 9 categorical independent variables, and 3 continuous independent variables, with a sample size of 1232. The 5 dependent variables are just different ways of ...
- 113
3
votes
1
answer
106
views
high variance proportion for intercept
I'm using the ols_eigen_cindex function to assess multicollinearity. With these variance proportions:
...
- 1,623
1
vote
1
answer
248
views
Fixed Effects causes Multicollinearity
I have a dataset that contains every tranche of a Deal (a Deal may contain multiple Tranches) and its Variables like Poolsize, Rating etc. The Tranches are included into the dataset at the time the ...
