assuming independency between independent variables in multiple regression? I heard that multiple regression assumes that the independent variables are 
correlated somehow. So when we convert the multiple regression into SEM diagram,
we see covariance arrows are drawn between independent variables.
So I wonder what if we assume the independecy between independent variables?
Does it change the estimates of coefficient? How much would it make difference?
And in what conditions would the difference get larger?
If possible, some mathematical background would be appreciated.
 A: The model for basic linear regression is $Y = X\beta + \varepsilon$ where $\varepsilon \sim N(0, \sigma^2 I_n)$. We model $E(Y|X)$. The $X$'s are considered fixed and without measurement error. There are 3 assumptions being made here. They are


*

*a linear model is correct

*the errors are normal with mean 0 and constant variance

*the errors are all independent.


Note that as a result of this $E(Y|X) = X\beta$ and $Var(Y|X) = \sigma^2I_n$. We also have that $\forall i \neq j \ \ Y_i \perp Y_j$. We could also formulate this by having $Var(\varepsilon) = W$ where $W$ is diagonal or $\Sigma$ where $\Sigma$ is some symmetric and positive definite matrix. These would each lead to different estimates of $\beta$ because $Var(\varepsilon)$ is a component of the solution to $\hat{\beta}$.
Absolutely no assumptions are made about $X$ other than that it is fixed and without measurement error. We may additionally want to assume that it is full rank and that $p < n$ but that's purely because of obtaining $\hat{\beta}$, not because of the model itself. The columns $X_i$ of $X$ can be highly correlated and might give us trouble with $(X^TX)^{-1}$ but that does not violate the model.
Does this address your question?
