I understand the concepts of Bayesian linear regression and regular neural networks separately, but I cannot wrap my head around how to combine both.
In a general setting, lets say I have a (deterministic) regular neural network, and I would like to build a Bayesian linear regression on the top of it and train them together. If it's not Bayesian linear regression, we can easily train them with backpropagation. But with two together, I do not understand how to train the neural network using gradient descent.
For my more specific problem setting, I have a set of vectors, and I want to first apply the same linear transformation to each of them, and then find the best linear combination of them to fit a target vector. The linear transformation is a matrix that maps the same dimension of the vectors to the same dimension. The linear combination part can be formulated as linear regression problem and the whole formula is as follows:
$\operatorname*{argmin}_{\theta, W} ||\theta^T(XW^T)-y||^2 $ where specifically, $X \in R^{m\times d}, W \in R^{d\times d}, \theta \in R^m, y \in R^d$ and $W$ is the transformation matrix and $\theta$ is the coefficient of the linear regression. I would like to pose prior on $\theta$ and thus make it Bayesian, but let $W$ remain deterministic.
Maybe it's because of my lack of understanding in Bayesian networks and other knowledge, I have searched up the internet but cannot find the right contents I want. I may be over-complicating things too. Please give me a direction or keywords that points to the solution of my problem. Thank you very much.