Orthogonal predictors (single regression coefficients aren't the same as multiple regression coefficients)? Let's say we have 2 predictors $X_1, X_2$. And there are $n=5$ samples. Say we have the following values for $X_1, X_2$:
$$
(1, -5) \\
(2, 0) \\
(3, 4) \\
(4, -3) \\
(5, 1)
$$
$X_1$ and $X_2$ here are clearly orthogonal. If we include the intercept and introduce a column of ones, we find that for $y = [0,1,2,3,4]$
$$
\hat{\beta} = [-1, 1, 0]
$$
which are the intercept, coefficient corresponding to $X_1, X_2$, respectively.
Now if I did simple linear regression for Y vs $X_1$ and $Y$ vs $X_2$. I find that $\hat{\beta}_1 = 1$, but $\hat{\beta}_2 = 0.18292683$. Why are they different here?
For orthogonal predictors shouldn't the single regression coefficients be the same as the multiple regression coefficients?
 A: 
For orthogonal predictors shouldn't the single regression coefficients be the same as the multiple regression coefficients?

Yes, but only if you don't fit an intercept:
> X1 <- c(1,2,3,4,5)
> X2 <- c(-5,0,4,-3,1)
> 
> y <- c(0,1,2,3,4)
> 
> lm(y ~ -1 + X1 + X2) %>% summary()
Coefficients:
   Estimate Std. Error t value Pr(>|t|)   
X1  0.72727    0.06663   10.91  0.00165 **
X2  0.05882    0.06920    0.85  0.45774   

> 
> lm(y ~ -1 + X1) %>% summary()
Coefficients:
   Estimate Std. Error t value Pr(>|t|)    
X1  0.72727    0.06428   11.31 0.000348 ***

> 
> lm(y ~ -1 + X2) %>% summary()

Coefficients:
   Estimate Std. Error t value Pr(>|t|)
X2  0.05882    0.38235   0.154    0.885


A: For what you said to be true all the time, your predictors should not only be orthogonal to each other, but they must also be orthogonal to the vector of 1s.
What this means is you need to look at the centered version of each predictor (when you center a vector, you are orthogonalizing it with respect to the vector of 1s), and assess orthogonality (in this case, you're assessing uncorrelatedness since uncorrelated means the same thing as orthogonality for centered vectors/predictors).
Note also that $\langle X_1, X_2 \rangle = 0$ does NOT necessarily mean $\langle X_1 - \bar{X_1}, X_2 - \bar{X_2} \rangle = 0$, as you can observe in your case. If they are, then you can individually regress $y$ on $1, X_1 - \bar{X}_1, X_2 - \bar{X}_2$ INDIVIDUALLY and obtain $\hat{\beta}_0, \hat{\beta}_1, \hat{\beta}_2$, which will be equivalent to the coefficients determined from regressing on all 3 variables simultaneously. In addition the  $\hat{\beta}_1, \hat{\beta}_2$ will be equivalent to the coefficients determined from multiple regression of $y$ on $1, X_1, X_2$ (uncentered versions), but $\hat{\beta}_0$ will be different.
This is easy to see when you examine
$$
\hat{\beta} = (X^TX)^{-1}X^TY
$$
If $X_1 \perp X_2 \perp 1$, then the columns of $X$ are orthogonal, and hence $X^TX$ is a diagonal matrix. The inverse of a diagonal matrix is diagonal. So each coefficient will have no influence on another.
