Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Full model significant but insignificant predictors (NO COLLINEARITY) [duplicate]

A follow-up on this unceremoniously deleted question, and before you yet again send me to this one, keep in mind that, as was re-iterated in the original formulation, collinearity is not an issue here ...
0
votes
0answers
6 views

Theoretical Results Regarding the Sum of OLS Coefficients

Consider the following regression model $$ y=\beta _{1}x_{1}+\beta _{2}x_{2}+u $$ where $x_{1}$ and $x_{2}$ are two random variables and $u$ is a disturbance term. I simulate the model drawing $x_{1}$,...
0
votes
1answer
27 views

What do you call a situation where in a regression the same variable appears in both the left- and right-hand side of the equation?

For example: GDP = B0_constant + B1_(GDP/Pop) + B2_X2 + B3_X3 Given that GDP appears on both left- and right-hand side this must certainly be problematic. What is this particular situation called?
1
vote
0answers
34 views
+50

Can you control for confounding variables by building a seperate primary and residual model?

TLDR: I want to seperate out any possible influence from a set of confounding variables on the response before estimating the effects of the variables of interest. Can I use two sequential modles? I ...
1
vote
0answers
36 views

Assessing potential multicollinearity for various model types meant for causal inference [duplicate]

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar ...
1
vote
0answers
17 views

How to deal with multicollinearity on categorical dataset with ordinal and nominal variables in ordinal Logistic regression to get odds ratios?

My dataset consists of ordinal and nominal variables for independent variables and an ordinal variable for the dependent variable. I have been trying to get odds ratios using the coefficients of ...
1
vote
1answer
21 views

How do I correctly treat nested variables in a regression given multicollinearity of said variables?

As per the question, I want to run a regression of variables where those variables are nested within each other and therefore highly correlated. Here is my specific example for context: I study the ...
0
votes
0answers
10 views

How to analyze data from unbalanced fractional factorial design

I ran a 2x2x3 mixed fractional factorial design that collected reaction times for detecting targets across depths. Within-subject IVs: Cue depth (levels: near, mid, far) Depth validity (levels: ...
1
vote
0answers
22 views

Regression with predictors that are intrinsically correlated

I am trying to analyze the effect of different land-use cover predictors (% of land area) on my response of interest. The land-use predictors cover the dominant types of land-use and hence ...
0
votes
0answers
9 views

Problems with multicollinearity tests of no intercept models

i have an $\mathrm{AR}(1)$ model in R, $y_t \sim y_{t-1} + X_{1,t}+ \cdots+ X_{n,t}$ with no intercept and i want calculate some individual multicolinearity tests for each variable in the model. The ...
1
vote
2answers
48 views

Why does removing the constant term prevent the dummy variable trap?

I understand that if you have a dummy variable with $m$ categories that you should include $m-1$ categories in order to avoid perfect collinearity between regressors. However I don't understand why ...
0
votes
0answers
10 views

Squared variable omitted in estimation of Prais-Winsten and Cochrane-Orcutt

I have estimated a model with OLS, where I have found autocorrelation with Durbin-Watson and Breusch-Godfrey. I want to use Prais-Winsten or Cochrane-Orcutt to remove the problem. When estimating the ...
0
votes
0answers
16 views

Variance Inflation Factor calculation problem

I am posted with this question that I really need help! A regression model includes 10 explanatory variables; the value of R squared for explanatory variable X1 with respect to other 9 variables is 0....
1
vote
0answers
34 views

Combining correlated predictors via binomial logistic regression

I'm new to the site and have tried to answer my question by reading old queries but it's a bit specific and I haven't been able to work it out - apologies for any unnecessary duplication if I've ...
0
votes
0answers
17 views

Coefficient omitted due to time fixed effect and multi-collinearity

I am working on a research paper (my first one) where I am dealing with company level unbalanced panel data. There are broadly two types of control variables in the study - a) firm level control ...
0
votes
1answer
25 views

Multivariate logistic regression not returning expected significant p-values in R

I am very new to R and statistics in general and have been stuck on this for a couple of weeks so any input would be greatly appreciated. I have a binary outcome variable ...
-2
votes
1answer
33 views

How does the covariance matrix of the predictors in multiple regression relate to the matrix $(\mathbf{X}^T \mathbf{X})^{-1}$? [duplicate]

Note in advance, due to my question previously being marked as a duplicate, that the question I ask here is concerned with the relationship between $(\mathbf{X}^T \mathbf{X})^{-1}$ and $\text{Cov}[\...
0
votes
0answers
24 views

What is the relationship between $\text{Cov}[\mathbf{X},\mathbf{X}]$ and $(\mathbf{X}^T \mathbf{X})^{-1}$ in multiple regression? [duplicate]

The matrix formulation of multiple regression for $n$ observations is $$ \mathbf{Y} = \mathbf{X}^T \beta + \varepsilon, $$ where the error $\varepsilon$ has finite variance $\sigma^2$. Let $\mathbf{b}$...
0
votes
1answer
25 views

How can statistical inference be done when using multicollinear data?

I want to model a variable $y$, which is known to be affected by a set of 5 variables: {$x_1$, $x_2$, $x_3$, $x_4$, $x_5$}. These variables are known to be correlated with one another and $y$ to some ...
0
votes
0answers
16 views

Goodness-of-fit for conditional logistic regression in a 1:1 matched case-control study

Dear Stackexchange community; I would appreciate if someone would guide me on this matter. On data analysis of a 1:1 matched case control study based on age and gender through using conditional ...
0
votes
0answers
34 views

Backwards stepwise model building in R

I have been working my way through some data to build a logistic model. I screened variables (most of them categorical) through an unconditional analysis, letting variables with a p-value of <0.2 ...
0
votes
0answers
8 views

Difference between collinearity and interaction effect with example?

I came across a great answer to the difference between collinearity and interaction effect here: what is the difference between collinearity and interaction? To help my understanding even further, I ...
0
votes
0answers
12 views

Multiplying regressors in linear regression model and multicollinearity

I am trying to train a linear regression model on $n$ data samples. My training set contains two features sets, $X_i$ and $Y_i$ for $i \in \{1, 2, \dots, n\}$. I want to create a transformed third ...
1
vote
0answers
53 views

How to use causality knowledge to improve linear regression? [closed]

As Peter mentioned in this reply https://stats.stackexchange.com/a/26412/152503 Sometimes we do have a priori information about the causality relationship between features/predictors Using the fire ...
1
vote
0answers
40 views

Logistic regression when most of variables are strongly correlated

As one of my assignments for uni I need to create a logistic regression model using the breast cancer dataset which is available here: data I've looking through the data and I see that I have an issue ...
0
votes
1answer
39 views

What to fix first when testing for multicollinearity, autocorrelation and heteroskedasticity?

I have a Twoways "within fixed effects panel regression model and detected multicollinearity, autocorrelation and heteroskedasticity. For heteroskedasticity I want to use heteroskedasticity-...
0
votes
0answers
5 views

Sequential anova to deal with collinearity?

I want to see whether animals with a larger brain size (for their body size) also have a larger gut length. The issue is that both brain size and gut length are highly correlated with body size (R-...
0
votes
1answer
23 views

Multiple Regression Multicollinearity issue

So using the multiple regression model, I am working on finding the relationship between several x factors and the average life expectancy at birth. x1= adult education level (age 25-64) x2= infant ...
0
votes
0answers
27 views

Testing for Collinearity in a Dataset with Categorical and Continuous Variables

I have a dataset that has 17 variables. 9 categorical and 8 continuous. Some have more than 2 levels. I've reduced the dimensionality significantly. I am looking for strategies to test for colinearity ...
0
votes
2answers
91 views

What's the difference between multicollinearity and autocorrelation?

I am learning about MLR and I don't quite understand the difference between these two terms? Could someone explain it to me? Additionally, if I made a model and I wanna validate the conditions ...
0
votes
0answers
50 views

How to treat Correlated covariates in linear regression analysis?

I have multiple covariates shown on the plot. AGE and DIABDUR are strongly correlated. To proceed with regression analysis do I include only one of those two covariates, AGE or DIABDUR or maybe do I ...
1
vote
0answers
39 views

R - dealing with new aliased coefficient (“NA” coefficient) in categorical variables for VIF

Currently, I want to check multi-collinearity among different categorical variables. FYI, I'm using 2 independent variables - category and ...
8
votes
1answer
119 views

Why not use PCA in every linear regression setting to avoid multicollinearity?

I realise the stupidity of this question, but hear me out. Imagine a linear regression (e.g. OLS) setting where we perform PCA on all of our independent variables and use all of the resulting ...
4
votes
1answer
52 views

Dealing with multi-collinearity in fixed and random effects for linear mixed models [lme4]?

I have a dataset where multiple doctors assessed a score for three experimental conditions, they also saw each patient from three different angles. I am now trying to predict the assessment by the ...
3
votes
0answers
99 views

“Regression design matrix is rank deficient” - multicollinearity between two categorical variables?

I have the following model: Reduction_in_clinical_score ~ Baseline_clinical_score + Site_of_data_collection + Treatment_Type + Age + Sex + ERP Site of data ...
0
votes
1answer
24 views

scikit-learn linear regressor digests perfectly collinear features?

I am currently running this little piece of code: ...
0
votes
0answers
30 views

Index Breakdown

Suppose I created an index, $I$, such that: $$I = aX + bY + cZ$$ If I regressed I against another variable such that: $$B = dI + eA$$ Would rewriting $B$ as $$B= d(aX + bY + cZ) + eA$$ allow us to ...
1
vote
0answers
47 views

Creating a correlation matrix for categorical variables in R

I am new to R and have been trying to solve this problem by myself for many hours without success. I am conducting logistic regression and have used the glm() function for univariable analysis. I have ...
0
votes
1answer
19 views

Issues in testing a simple linear relationship. Collinearity? Misspecification? Any other insight?

I have a theoretical model saying that Y should be equal to: Y = X + c * (W - X) + (Z1 - Z2), where c is a given constant. Here, it may be important to say that X is measured with error. Someone ...
2
votes
1answer
44 views

How to deal with predictors which are not significant, although r-squared is significant?

I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation ...
1
vote
1answer
37 views

How does PLSR solve Multicollinearity

We know that PLSR is a very common way to solve Multicollinearity in the Multiple Linear Regression. But do you know how does it work in detail? And why Multicollinearity of $x$ will be related to the ...
0
votes
0answers
14 views

contradiction between PCA and Multicollinearity

From here Should one remove highly correlated variables before doing PCA? we know that when there are some highly correlated Features during the PCA, we should remove them to avoid some incorrect ...
1
vote
1answer
30 views

multicollinearity and interaction

I am a bit confused between multicollinearity and interaction. Would anyone mind explaining it to me and the difference between the two and how they affect the variables? I understand that ...
1
vote
1answer
32 views

Evidence that Collinearity Causes Predictors to be Insignificant

Question: Is there any evidence that collinearity causes some predictors in this model to be insignificant? Using R, I calculated the correlations of the predictor variables of a linear model. ...
0
votes
0answers
25 views

Using model comparison to deal with collinearity in linear mixed models

I am working on a set given to me, which involves fitting logistic models with a couple of predictors, some of which are nearly perfectly correlated (.9), imagine, face features deviations of specific ...
0
votes
1answer
14 views

Anova for potential colinear data

I have 4 columns of data, 1. Time (so time at 0, 6, 12, 24), 2. Drug types (A vs B), 3. Gene types (X, Y, Z) and 4. Values Now I’m trying to find whether drug A differs than drug B taking into account ...
1
vote
1answer
27 views

small T and large N, problem of collinearity in the model

If I am taking the log of a variable $X$ and also a square term of the variable $X$ on the right-hand side. Say, the equation is $$\log Y_{it} = α + β_1\log X + β_2 \log X^2 + u_i$$ where $Y =$ Waste ...
3
votes
1answer
59 views

Multiple Linear Regression with more variables than samples

I'm currently learning chemometrics for my work and I have a simple question about Multiple Linear Regression (MLR). Just to explain the context: I am simply using UV-Vis-NIR spectra (2500 wavelengths)...
0
votes
0answers
30 views

Logistic Regression with dummy variables? [duplicate]

I am working on a problem where response variable is binary and my features are dummy variables. I observed when I include intercept to model all the dummy variables' p-values are equal to 1. When I ...
0
votes
0answers
17 views

Multicollinearity in multiple linear regression with only categorial variables

I have to do a multiple linear regression with a numeric dependent variable and three categorial variables (2x2x4) as independent variables. Do I have to check for multicollinearity and if so, why and ...

1
2 3 4 5
20