Let's say Im trying to do a bayesian linear regression and I have some strong prior on the parameters x1,...,xn.
However, after collecting my data I can clearly see the residuals are not normally distributed, and thus can't calculate the likelihood correctly.
What should I do? Use a transformation?
Thanks
So you are confusing two sources of randomness in Bayesian linear regression. (1) Randomness over the parameter space, and (2) Randomness in the input noise
Assuming your aim is to model something like $Y = \beta X + \varepsilon$, you can have $\varepsilon \sim \mathcal{N}(0,\sigma^2)$, so the noise is normally distribution, and then also $\beta \sim \mathcal{N}(\mu_{\beta},\sigma_{\beta}^2)$, so then the parameters are distributed according to a normal distribution also.
Thus your prior over your parameters $p(\beta)$, and your distribution of residuals $p(\varepsilon)$ are two different and independent modelling assumptions, and the choice of one model, in most cases does not effect your choice of the other model.
Now, onto the other part of your question. In the framework of Bayesian linear regression, it is possible to for your noise to look like any distribution you want. If your noise is not normal you can reflect that in $p(\varepsilon)$.
However, in general a normality assumption in the noise (and parameters) just makes things much easier to calculate, and evaluate in post processing. It is a very well-understood system. So much so, that in general if your noise does not look Gaussian it would be advisable to apply a transformation to it so it appears Gaussian. Look at logarithmic transformations, and Box Cox transformations for this.
I strongly looking at the following post also, in regards to noise:
Regression when the OLS residuals are not normally distributed
