Using a Bayesian method, you would treat each possible model as a parameter and marginalize out the parameters. You would need to set priors over the model space as well.
In the most extreme case, it would be an entirely combinatoric exercise. If you can logically restrict the combinations, you can reduce the number of hypotheses.
Let me give you an example, the Current Ratio, and the Quick Ratio are almost identical in finance. If you were testing financial ratios as an element of your model, they should not both be in your model as there is little extra independent information being generated by having them both. Their shared information is so high that even if having both were a better fit the added computational cost of having them both is high because Bayesian posteriors won't double count information while Frequentist estimators will.
So if you are not going to solve it in a combinatoric manner, then you should be thinking in terms of shared information and not variables. The only difficulty with logically reducing your set is that your inference cannot be considered complete because there will be possible ways in which the world is constructed that you have ignored.