MSE decomposition to Variance and Bias Squared In showing that MSE can be decomposed into variance plus the square of Bias, the proof in Wikipedia has a step, highlighted in the picture. How does this work? How is the expectation pushed in to the product from the 3rd step to the 4th step? If the two terms are independent, shouldn't the expectation be applied to both the terms? and if they aren't, is this step valid?
 A: There has been some confusion about the question which was ambiguous being about the highlight and the step from line three to line four.
There are two terms that look a lot like each other. 
$$\mathbb{E}\left[\hat{\theta}\right] - \theta \quad \text{vs} \quad \mathbb{E}\left[\hat{\theta}\right] - \hat\theta$$
The question, about the step from 3rd to 4th line, relates to the first term:


*

*$\mathbb{E}[\hat{\theta}] - \theta$ this is the bias for the estimator $\hat\theta$
The bias is the same (constant) value every time you take a sample, and because of that you can take it out of the expectation operator (so that is how the step from the 3rd to 4th line, taking the constant out, is done).
Note that you should not interpret this as a Bayesian analysis where $\theta$ is variable. It is a frequentist analysis which conditions on the parameters $\theta$. So we are computing more specifically $\mathbb{E}[(\hat{\theta} - \theta)^2 \vert \theta]$, the expectation value of the squared error conditional on $\theta$, instead of $\mathbb{E}[(\hat{\theta} - \theta)^2]$. This conditioning is often implied implicitly in a frequentist analysis.
The question about the highlighted expression is about the second term


*

*$\mathbb{E}[\hat{\theta}] - \hat{\theta}$ this is the deviation from the mean for the estimator $\hat{\theta}$. 
It's expectation value is also called the 1st central moment which is always zero (so that is how the highlighted step, putting the expectation equal to zero, is done).
A: $E(\hat{\theta}) - \theta$ is not a constant.
The comment of @user1158559 is actually the correct one:
$$
E[\hat{\theta} - E(\hat{\theta})] = E(\hat{\theta}) - E[E(\hat{\theta})] 
= E(\hat{\theta}) - E(\hat{\theta}) = 0
$$ 
A: The trick is that $\mathbb{E}(\hat{\theta}) - \theta$ is a constant.
