# Bayesian linear regression KL divergence

$$y_i \sim N(w_0 + w_1x_i, \sigma^2_j)$$ $$\mathbf{w} \sim N(0,\alpha^2 I)$$ Data is $$D$$, posterior distribution $$p(\mathbf{w}|D)$$ is approximated according to mean-field approximation $$p(\mathbf{w}|D) \approx \prod_{d=0}^1 q(w_d)= \prod_{d=0}^1 N(w_d | \mu_d, \sigma^2_d)$$

How to calculate the KL divergence of: $$\text{KL} \left[q_{\lambda}(\mathbf{w}) | p(\mathbf{w})\right]$$

I know that analytical solution of KL for two univariate gaussians would be: $$-\frac{1}{2} + \log(\frac{\sigma_1^2}{\sigma_2^2}) + \frac{\sigma_2^2 + (\mu_1-\mu_2)^2}{2\sigma_1^2}$$ But not sure what to do here since $$p(\mathbf{w}|D)$$ itself has two components.

You can use the formula for KL divergence between two multivariate normal distributions: $$D_{KL}(\mathcal N_0\vert\vert N_1)=\frac{1}{2}\left(\text{tr}(\Sigma_1^{-1}\Sigma_0)+(\mu_1-\mu_0)^T\Sigma_1^{-1}(\mu_1-\mu_0)-k+\ln\left(\frac{\det\Sigma_1}{\det\Sigma_0}\right)\right)$$
In your case, $$\mathcal N_0=\mathcal N \left(\mathbf{\mu_0}=\begin{bmatrix}\mu_0\\\mu_1\end{bmatrix}, \mathbf{\Sigma_0}=\begin{bmatrix}\sigma_0^2&0\\0&\sigma_1^2\end{bmatrix}\right)$$, and $$\mathcal N_1=\mathcal N( \mathbf{\mu_1}\mathbf = 0, \mathbf{\Sigma_1}=\alpha^2\mathbf I)$$, and $$k=2$$. Note the differences between the bold and normal fonts, i.e. $$\mathbf{\mu_0}\neq\mu_0$$.