Understanding how the prior affects the estimate slope in Bayesian regression I was reading this article on characterizing the posterior for a simple Bayesian linear regression (Gaussian posterior) when I saw the posterior mean and variance described thusly:
\begin{split}
\mu_n &= (X^TX+\sigma^2\Lambda)^{-1}X^Ty,\\
\Sigma_n &= \sigma^2(X^TX+\sigma^2\Lambda)^{-1}.
\end{split}
where $\Lambda$ is the precision matrix. 
Here is my question: does the term $\sigma^2 \Lambda$ only make the slope smaller? What exactly is going on here?
 A: As the article describes, the posterior you wrote is the result of putting a zero-mean Gaussian prior on the weights. The consequence of this is, indeed, that the weights will be smaller than they would be for maximum likelihood/least squares estimation.
One way to understand this is by considering the shape of the Gaussian prior. It assigns greater probability to weight vectors closer to the mean (i.e. zero). Compare this to the maximum likelihood/least squares case, where the equivalent prior is the improper uniform prior that gives no preference to any weight vector over any other.
Another way to understand this is by thinking about ridge regression. Here, the goal is to find weights that reduce the squared error, but a penalty is placed on the magnitude of the weight vector. The effect of this penalty is to shrink the weights toward zero. The loss function is:
$$\sum_{i=1}^n (y_i - x_i \beta)^2 + \lambda \|\beta\|_2^2$$
where $\beta$ is the weight vector, $x_i$ is the $i$th data point, $y_i$ is the $i$th target value, and $\lambda$ controls the strength of the penalty. As $\lambda$ increases, the weights are increasingly shrunk toward 0. It turns out that ridge regression is equivalent to MAP estimation using a zero-mean Gaussian prior on the weights, whose covariance matrix is a scaled version of the identity matrix (this case is mentioned in the article you linked). The scale of the prior is controlled by $\lambda$--larger penalties correspond to priors that are more strongly concentrated around 0.
