Linked Questions

2
votes
0answers
99 views

multicollinearity in OLS regression [duplicate]

If I have a dependent variable Y and two independent variables X1 and X2 , that are highly correlated. Y ~ beta1*X1 + beta2*X2 What issues can multicollinearity cause in an OLS regression, apart ...
66
votes
1answer
68k views

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
31
votes
3answers
45k views

Regression coefficients that flip sign after including other predictors

Imagine You run a linear regression with four numeric predictors (IV1, ..., IV4) When only IV1 is included as a predictor the standardised beta is +.20 When you ...
11
votes
6answers
3k views

How problematic is it to control for non-independent covariates in an observational (i.e., non-randomized) study?

Miller and Chapman (2001) argue that it is absolutely inappropriate to control for non-independent covariates that are related to both the independent and dependent variables in an observational (non-...
6
votes
3answers
7k views

Is centering a valid solution for multicollinearity?

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and ...
8
votes
3answers
4k views

Testing difference between two (adjusted) r^2

Let's say I have two regression models, one with three variables and one with four. Each spits out an adjusted r^2, which I can compare directly. Obviously, the model with the higher adjusted r^2 is ...
3
votes
2answers
15k views

Relationship Between Correlation and Multicollinearity [duplicate]

Suppose I've a model such as $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k + \epsilon$. Now, there's high correlation between $X_1$ & $X_2$ (say over 60% but below 75%). Does ...
5
votes
4answers
4k views

Modeling prices with the Hedonic regression

I'm using the concept of Hedonic regression in order to model the prices for real estates. I'm having some trouble with my approach. What I have and what I do my data consists out of real estates ...
1
vote
1answer
871 views

Fitting linear model through noisy data

I'm currently working on a predictive modeling project. I have to predict $Y$ given variables $X_1,X_2,X_3$ and $X_4$ that are not necessarily independent. Our first idea was to propose a linear ...
6
votes
1answer
757 views

Abusing Linear Models under Multicollinearity: Simulation for 'realistic' movement of predictors

I have a reasonable understanding of why multicollinearity is a problem is regression models, along the lines of this excellent post. To summarise my understanding, for a regression model of $y = \...
1
vote
2answers
319 views

partial correlation to control for a continuous variable when examining the relationship between continuous and categorical DV and IVs

I have two similar issues. I have two separate questions, one has a binary DV (is there a brain response yes/no) and the other a continuous DV (how widespread the response is). Each question is ...
1
vote
0answers
125 views

Do control variables in a regression analysis cause collinearity?

This is something that bothers me for quite some time, but I didn't find yet a satisfactory answer. I hope that the wisdom of the people hear will help me to clarify this: In a multivariate ...