From what you’re saying is that you want something Bayesian, but you can’t define the likelihood. For such cases there is [approximate Bayesian computation](https://en.wikipedia.org/wiki/Approximate_Bayesian_computation) (see [tag:abc]), where in place of likelihood you use some summary statistics.

As a side note, for using proper Bayesian analysis you don’t need to know the “exact” distribution. We nearly never do. You need to use some distribution that relatively well approximates the distribution of the data. This is how is it done in most of the statistics. We don’t use Gaussian, Poisson, etc distributions because they are the exact distributions of the observed data, but they are good enough approximations for the purpose. Same you do with loss function, you don’t use squared error because it has some deep meaning for your data, but because it works well enough.