The Solomonoff universal prior is fixed relative to a specific choice of universal Turing machine (UTM). Now, I understand that a UTM can simulate any other UTM, so that they assign a complexity to any bitstring that differs by at most a constant.
Doesn't this invalidate the whole point of the universal prior? Haven't we introduced bias by choosing one UTM versus another? Some machines will be simpler to define relative to one UTM than another. And while two UTMs will only disagree on the complexity of a bitstring by at most a fixed amount, that amount depends on the UTM pair and can be arbitrarily great.
One advantage to the Solomonoff universal prior is that it is over a model class that truly includes all computable models, which is about as large a model domain as we can hope for. And with enough data the true model should be assigned high probability in the posterior (assuming it is computable).
However, isn't it also true that any prior probability distribution that assigns non-zero probability to every computable model will also be able to converge to the true model?
In fact: won't such a prior distribution implicitly define a UTM?
Have we accomplished anything here?