1
$\begingroup$

I am curious whether the property of homoskedasticity is more or less dependent on the correlation between independent variables. I assumed that if the $cor(x_1,x_2....x_n) \approx 0$, hence the movements of each variable became a random path. But on the other hand we also do not want to perfect collinearity between independent variables, since it prevents us to calculate the marginal affects of each indepent variable on dependent variable. So, if the correlation between independent variables are high, I guess the homoskedasticity assumption will be more likely to be observed. But, a high correlation also means that, it makes it hard to measure the effects of each independent variable on endogenous variables. Is there any percentage which gives us the best result when it comes to the homoskedasticity and the explanation part? Or correlation between independent variables should not be considered deeply?

$\endgroup$
7
  • $\begingroup$ I would have thought these were different issues in linear regression $\endgroup$
    – Henry
    Nov 17, 2023 at 12:43
  • $\begingroup$ Shawn gave a very complete answer. I'm just curious where you got the notion that they should be related and what "hence the movements of each variable became a random path." means. $\endgroup$
    – Peter Flom
    Nov 17, 2023 at 13:59
  • $\begingroup$ First of all, my intuition on the homoskedasticity was wrong. As the independent variables become correlated, then the standard error of regression would increase, thus, I guess, homoskedasticity will be less probable, since $dof$ decreases while the model does not improve (Maybe, this is also untrue). However, I don't think that the conclusion of $corr(x_1,x_2...x_n)=0$ from sample data is a good idea, then how are we going to assume perfect collinearity between independent variables? $\endgroup$ Nov 17, 2023 at 17:31
  • $\begingroup$ @TunaySabriYüksel What does $corr(x_1,x_2,\dots,x_n)$ mean? $\endgroup$
    – Dave
    Nov 17, 2023 at 17:39
  • $\begingroup$ And in addition to that, I think I have to answer the reason why "they should be related". Consider that a cross-sectional data with thousands of observations. It is has a very low probability that the correlation between two random variable is going to be 0. If the correlation is found as 0, then I think, we should interpret it as one of the variable purposely follow a path where the correlation is 0 with other random variable.Which makes 0 correlation unuseful. From my interpretation, 0 correlation actually means something different (rather than there no are linear relationship). – $\endgroup$ Nov 18, 2023 at 21:09

1 Answer 1

3
$\begingroup$

There really isn't anything saying that these two things are explicitly related. You can have two predictors that are:

  • Almost perfectly collinear with heterogenous variance
  • Almost perfectly collinear with homogenous variance
  • Nearly orthogonal with heterogenous variance
  • Nearly orthogonal with homogenous variance

I simulate all of these cases below in R.

Collinear with Homogenous Variance

#### Set Seed ####
set.seed(123)
library(performance)

#### Collinear No Var Issues ####
x1 <- rnorm(100)
x2 <- (4*x1) + rnorm(100)
y <-  (.4*x1) + (2*x2) + rnorm(100)
df <- data.frame(x1,x2,y)
pairs(df,main="VIF: 14.92")
fit <- lm(y ~ x1 + x2)
check_collinearity(fit)

enter image description here

Collinear with Heterogenous Variance

#### Collinear Var Issues ####
x1 <- rpois(100,3)
x2 <- (2*x1) + rnorm(100,sd=.1)
y <-  (.4*x1) + (.4*x2) + rnorm(100)
df <- data.frame(x1,x2,y)
pairs(df)
fit <- lm(y ~ x1 + x2)
check_collinearity(fit)

enter image description here

Uncorrelated with Homogenous Variance

#### Not Collinear No Var Issues ####
x1 <- rnorm(100)
x2 <- rnorm(100)
y <-  rnorm(100)
df <- data.frame(x1,x2,y)
pairs(df,main="VIF: 1")
fit <- lm(y ~ x1 + x2)
check_collinearity(fit)

enter image description here

Uncorrelated with Heterogenous Variance

#### Not Collinear Var Issues ####
x1 <- rbeta(100,20,1)
x2 <- rbeta(100,1,20)
y <-  rbeta(100,1,20)
df <- data.frame(x1,x2,y)
pairs(df,main="VIF: 1")
fit <- lm(y ~ x1 + x2)
check_collinearity(fit)

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.