0
$\begingroup$

I'm comparing the relative weights of features in predicting an effect, with the point that one is "significantly" higher than other ones. However, I'm not sure how solid it is to perform frequentist statistical testing on weight distributions (I'm in a biology lab and unfortunately reporting things like p-values are pretty much unavoidable when trying to publish in the field). Using Variational Inference it is ill-posed to do e.g. a t-test, since you're optimizing for the parameters of distributions and they're already "different" by construction. Using MCMC you could perform a t-test using samples, but you can make any difference "significant" with enough samples. So what, if any, would be a sensible way to compare two coefficient distributions? Thanks!!

$\endgroup$
0
$\begingroup$

If I'm interpreting your question correctly, you need some way to select between models with different coefficient distributions. There are some methods of Bayesian model selection and assessing fit, notably WAIC and LOOIC. You can implement these in STAN and JAGS and many other Bayesian software options.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy