Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Linear Model coefficient estimation

Let's assume that we have two linear models: Linear Model 1 (LM1) Y = B1X1 + B2X2 + e Linear Model 2 (LM2) Y = BX1 + e Depending on whether X2 is independent vs. positively correlated vs. ...
1
vote
0answers
7 views

Age matching in three group ANCOVA design

I have to compare two groups of patients who are defined based on their chronicity of illness, hence of two different age distributions. One group is younger on average and the other older. I would ...
0
votes
0answers
12 views

Does pairwise correlation and multicollinearity matter in dispersion and zeroinflation model of glmmTMB?

I'm using glmmTMB to calculate beta-binomial GLMMs with nested and crossed random intercepts. I have overdispersed, zero-inflated data (assessed with Dharma). I use continuous terms in the very ...
0
votes
1answer
29 views

LASSO/Ridge regression with adjustment for a covariate

I'm trying to address the following analysis problem in high-dimensional biological data. The setup is bulk gene expression data where multiple cell types (tumor and immune cells) can contribute to ...
0
votes
0answers
13 views

multicollinearity in regressions

I am currently investigating the effects of maternal education on child mortality. (1): The education variable is a dummy variable, 1=if mother received education, 0 otherwise. (2): The education year ...
0
votes
0answers
33 views

How do get rid of (1 not defined because of singularities) in R? [duplicate]

I'm analyzing data in R, I'm trying to see how some variables affect test scores (Value) of different countries. In the data, since there is different time periods for different countries I need to ...
1
vote
0answers
11 views

Strange, symmetric suppression between 3 IVs in multiple regression?

I have encountered an interesting phenomenon while working on some data derived from a(n admittedly badly constructed) questionnaire by a colleague, and I have no idea what to make of it ...
0
votes
1answer
21 views

How to use a regression model for what-if-analysis?

I train a linear regression model on historic sensor data and controlable variables to predict a quality metric. The R² of the final model y = b0 + b1 * x1 + b2 * x2 + b3 * x3 is quite good. Now it ...
0
votes
0answers
18 views

How do I fix the problem of collinearity, i think i am in the dummy variable trap [duplicate]

I am estimating the whether floods affect electoral outcomes and I am using a difference-in-difference estimator. I have run my probit model with reelected party as the dependent variable (1 if ...
-1
votes
0answers
45 views

What do these warnings mean? [duplicate]

New to statsmodels,so can't figure out the warnings ...
0
votes
0answers
11 views

Do guidelines pertaining to multicollinearity and overfitting in glmm pertain to population-level or subject-level data?

I've read guidelines when model building, such as to avoid overfitting keep at least a 10:1 ratio between the sample size of the lowest response variable class to the number explanatory variables, or ...
1
vote
0answers
17 views

R glm() does not converge / huge standard errors: collinearity? [duplicate]

I am trying to run a fixed effects logit-regression in R using glm() as follows glm(binary_outcome ~ as.factor(region)*as.factor(birth_cohort) + as.factor(region)*as.factor(gender)+as.factor(...
0
votes
2answers
30 views

Do variables with similar correlation coefficient values multicollinear?

For example I have two variables, X1 and X2, with which I calculate the Pearson Correlation Coefficients with a target Y. Say, if both of them result in 0.6 as the correlation coefficient, can I say ...
2
votes
0answers
34 views

Can I use compositional data in a GLM?

I have landing data (kg) of five species where I try to identify which factors may be contributing most strongly to the high catch. Before, I had 4 predictor variables (Depth, Chlorophyll, Temperature ...
0
votes
0answers
19 views

large-n ols simple linear regression assumption “no perfect linear collinearity”

Suppose I have a model: $$ y = \beta_{0} + \beta_{1}X + \epsilon $$ where X is a binary dummy, either 0 or 1. Suppose all other conditions are satisfied (linearity, $y_{i}$ $x_{i}$ iid, ...
0
votes
0answers
26 views

How to explain features' importance after PCA?

I had a dataset that was terribly haunted by colinearity. I carried out PCA on it to remove the colinearity. Let's assume I am going to build a linear regression model, which is easily explainable, on ...
1
vote
1answer
16 views

What are the pitfalls of including a continuous variable and a discretized version of the same variable within a regression?

Besides the issue of collinearity - making coefficients kinda useless and p-values non interpretable, I was wondering, what would be the issue of including a continuous variable and the discretized ...
6
votes
1answer
68 views

How and why do epidemiology and econometrics models handle multi-collinearity differently?

Linear multiple-regression models are often built differently according to discipline. In epidemiology, a linear multiple-regression may be fit to test associations using hypotheses about model ...
0
votes
1answer
29 views

Aliased coefficients in panelgranger with just one variable

I'm running the pgrangertest package from plm, and getting the following error: ...
2
votes
1answer
43 views

Multicollinearity: quadratic correlation between two independent variables in polynomial regression

Consider a polynomial regression of the form $y = α*x_1^2 + β*x_1 + ​ɣ*x_2$ My question is: how to deal with multicollinearity between $x_1^2$ and $x2$? Or in other words, how to control for a ...
0
votes
0answers
13 views

Adaptive Lasso and Multicollinearity

I have been reading through several sources trying to determine the ability of different methods in the presence of multicollinearity. I understand that OLS regression and the lasso do not perform ...
1
vote
1answer
52 views

Propensity scores with reverse casuality between variables

I have a dataset about university students and their educational success. I would like to investigate how student sex influences the chances of graduation, but sex correlates with many other ...
0
votes
0answers
10 views

Nonsensical negative coefficent with multiple regression, without multicollinearity

I am a bit confused with a result that came up with a multiple regression. I have these data X1 X2 Y 287 4769 9972 291 5674 9732 308 3994 7999 315 3554 7138 321 5044 7642 330 5336 7765 334 ...
0
votes
1answer
42 views

Why would the significance of coefficients change in a logistic regression after transforming a variable?

I have a logistic regression with three independent variables. The correlation coefficients between the three variables are: For two of the variables with correlation coef of 0.1, if you do a ...
1
vote
1answer
18 views

Is multicollinearity ever an issue in ensemble learning?

Suppose I have two models, A and B, and suppose B takes the output of A as one of its features. Now suppose that both models use at least some of the same features. Is there a potential ...
1
vote
1answer
40 views

Two-way fixed effects and collinearity

I am modeling the relationship of an independent variable $(x_t/z_i),$ where $t$ indexes time and $i$ individual with an outcome $y_{it}$. The log transformation of $(x_t/z_i)$ is prohibitively ...
1
vote
0answers
25 views

Multicollinearity in Logistic Regression - chi-sq tests vs correlation matrix (corrb in SAS)

I have categorical variables (some 0/1, some nominal and some ordinal) and I'm getting different answers when using the two different approaches for deciding if there's multicollinearity. To get the ...
1
vote
0answers
10 views

Is it like the concept of dummy trap doesn't exist is machine learning apart from Logistic Regression & Multiple Linear Regression?

While performing MLR & Logistic Regression model summary analysis I have seen the problem of perfect multicolinearity if we use one hot encoding without dropping a single feature. Is it necessary ...
0
votes
0answers
30 views

correlation of PCA components

I'm used to posting on stack overflow for code questions, so not sure about the rules here, but I have some trouble with a Principal components analysis, and I don't know where to find help. either I ...
0
votes
0answers
16 views

Full model significant but insignificant predictors (NO COLLINEARITY) [duplicate]

A follow-up on this unceremoniously deleted question, and before you yet again send me to this one, keep in mind that, as was re-iterated in the original formulation, collinearity is not an issue here ...
0
votes
0answers
11 views

Theoretical Results Regarding the Sum of OLS Coefficients

Consider the following regression model $$ y=\beta _{1}x_{1}+\beta _{2}x_{2}+u $$ where $x_{1}$ and $x_{2}$ are two random variables and $u$ is a disturbance term. I simulate the model drawing $x_{1}$,...
0
votes
1answer
31 views

What do you call a situation where in a regression the same variable appears in both the left- and right-hand side of the equation?

For example: GDP = B0_constant + B1_(GDP/Pop) + B2_X2 + B3_X3 Given that GDP appears on both left- and right-hand side this must certainly be problematic. What is this particular situation called?
3
votes
0answers
69 views

Can you control for confounding variables by building a seperate primary and residual model?

TLDR: I want to seperate out any possible influence from a set of confounding variables on the response before estimating the effects of the variables of interest. Can I use two sequential models? I ...
2
votes
0answers
41 views

Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
1
vote
0answers
20 views

How to deal with multicollinearity on categorical dataset with ordinal and nominal variables in ordinal Logistic regression to get odds ratios?

My dataset consists of ordinal and nominal variables for independent variables and an ordinal variable for the dependent variable. I have been trying to get odds ratios using the coefficients of ...
1
vote
1answer
24 views

How do I correctly treat nested variables in a regression given multicollinearity of said variables?

As per the question, I want to run a regression of variables where those variables are nested within each other and therefore highly correlated. Here is my specific example for context: I study the ...
0
votes
0answers
12 views

How to analyze data from unbalanced fractional factorial design

I ran a 2x2x3 mixed fractional factorial design that collected reaction times for detecting targets across depths. Within-subject IVs: Cue depth (levels: near, mid, far) Depth validity (levels: ...
1
vote
0answers
28 views

Regression with predictors that are intrinsically correlated

I am trying to analyze the effect of different land-use cover predictors (% of land area) on my response of interest. The land-use predictors cover the dominant types of land-use and hence ...
0
votes
0answers
9 views

Problems with multicollinearity tests of no intercept models

i have an $\mathrm{AR}(1)$ model in R, $y_t \sim y_{t-1} + X_{1,t}+ \cdots+ X_{n,t}$ with no intercept and i want calculate some individual multicolinearity tests for each variable in the model. The ...
1
vote
2answers
101 views

Why does removing the constant term prevent the dummy variable trap?

I understand that if you have a dummy variable with $m$ categories that you should include $m-1$ categories in order to avoid perfect collinearity between regressors. However I don't understand why ...
0
votes
0answers
18 views

Squared variable omitted in estimation of Prais-Winsten and Cochrane-Orcutt

I have estimated a model with OLS, where I have found autocorrelation with Durbin-Watson and Breusch-Godfrey. I want to use Prais-Winsten or Cochrane-Orcutt to remove the problem. When estimating the ...
0
votes
0answers
18 views

Variance Inflation Factor calculation problem

I am posted with this question that I really need help! A regression model includes 10 explanatory variables; the value of R squared for explanatory variable X1 with respect to other 9 variables is 0....
1
vote
0answers
34 views

Combining correlated predictors via binomial logistic regression

I'm new to the site and have tried to answer my question by reading old queries but it's a bit specific and I haven't been able to work it out - apologies for any unnecessary duplication if I've ...
0
votes
0answers
43 views

Coefficient omitted due to time fixed effect and multi-collinearity

I am working on a research paper (my first one) where I am dealing with company level unbalanced panel data. There are broadly two types of control variables in the study - a) firm level control ...
0
votes
1answer
25 views

Multivariate logistic regression not returning expected significant p-values in R

I am very new to R and statistics in general and have been stuck on this for a couple of weeks so any input would be greatly appreciated. I have a binary outcome variable ...
-2
votes
1answer
51 views

How does the covariance matrix of the predictors in multiple regression relate to the matrix $(\mathbf{X}^T \mathbf{X})^{-1}$? [duplicate]

Note in advance, due to my question previously being marked as a duplicate, that the question I ask here is concerned with the relationship between $(\mathbf{X}^T \mathbf{X})^{-1}$ and $\text{Cov}[\...
0
votes
0answers
36 views

What is the relationship between $\text{Cov}[\mathbf{X},\mathbf{X}]$ and $(\mathbf{X}^T \mathbf{X})^{-1}$ in multiple regression? [duplicate]

The matrix formulation of multiple regression for $n$ observations is $$ \mathbf{Y} = \mathbf{X}^T \beta + \varepsilon, $$ where the error $\varepsilon$ has finite variance $\sigma^2$. Let $\mathbf{b}$...
0
votes
1answer
30 views

How can statistical inference be done when using multicollinear data?

I want to model a variable $y$, which is known to be affected by a set of 5 variables: {$x_1$, $x_2$, $x_3$, $x_4$, $x_5$}. These variables are known to be correlated with one another and $y$ to some ...
0
votes
0answers
24 views

Goodness-of-fit for conditional logistic regression in a 1:1 matched case-control study

Dear Stackexchange community; I would appreciate if someone would guide me on this matter. On data analysis of a 1:1 matched case control study based on age and gender through using conditional ...

1
2 3 4 5
20