# How to understand bayesian inference in the framework of deeplearning?

It is said that $$p \left( \theta | y _ { 1 : N } \right) \propto _ { \theta } p \left( y _ { 1 : N } | \theta \right) p ( \theta )$$.

And $$p \left( \theta | y _ { 1 : N } \right)$$ is the posterior, $$p \left( y _ { 1 : N } | \theta \right)$$ is the likelihood, and $$p ( \theta )$$ is the prior.

Suppose we have a model M, and its weights are W. To my understanding:

1. As for $$p \left( \theta | y _ { 1 : N } \right) \propto _ { \theta } p \left( y _ { 1 : N } | \theta \right) p ( \theta )$$, I think $$\theta$$ is $$W=\{w_1, w_2, ..., w_n\}$$, $$p(\theta)$$ is a distribution for $$w_i$$, $$p \left( y _ { 1 : N } | \theta \right)$$ is the accuracy, and $$p \left( \theta | y _ { 1 : N } \right)$$ is the ideal weights distribution. Are my understandings correct?

2. As for bayesian inference for $$p \left( \theta | y _ { 1 : N } \right)$$, first we select a distribution $$q(.)$$ for $$w_i$$, and the goal is to find the parameters(mean, variance, ...) for this distribution so that it is very close to the ideal weights distribution $$p \left( \theta | y _ { 1 : N } \right)$$. Using KL-divergence

$$\mathrm { KL } ( q ( \cdot ) \| p ( \cdot | y ) ) := ... := \log p ( y ) - \int q ( \theta ) \log \frac { p ( \theta , y ) } { q ( \theta ) } d \theta$$ , it is down to maximum $$\int q ( \theta ) \log \frac { p ( \theta , y ) } { q ( \theta ) } d \theta$$. But how to solve this? I see some materials talking about SVI, but still do not understand this well. Where can I find some codes for SVI? |......| If me, I will first use Adam to optimize all the $$w_i$$, and then recompute the mean and variance for the distribution $$q(.)$$, after resampling, use Adam again to optimize these new $$w_i$$, ... . Is this a feasible method?