In this blog by William Briggs, who seems to be a prolific lecturer/writer on pop-statistics, he condemns the "Bayesian Metaphor" which is essentially referring to Bayesian probability as belief.
Quoting from a post on vampires, “In Bayesian inference, you start with some initial beliefs (called ‘Bayesian priors’ or just ‘priors’), and then you ‘update’ them as you receive new evidence.”
This is the standard metaphor, and it’s not so much wrong as unhelpful, misleading, and restricting. The metaphor derives from Bayes’s rule (details which can be looked up anywhere) and which gives a formula which on the right-hand-side is supposed to be an element representing “prior beliefs.” The formula itself is correct, as most math is. But because math is correct does not mean that it means what you think it means.
In my opinion, the prior being a "belief" is not a metaphor, but it is a veritable reflection of Bayesian probability. For instance, Bayesians can probabilistically quantify events that frequentists cannot such as the location of a quantum particle. I understand the distinction between Bayes' Rule and Bayesian Statistics: the former is a probabilistic law that applies to frequentist as well as Bayesian inference as well; the latter deals with a Bayesian interpretation of probability. He goes on to say:
This is wrong because there is no such thing as “Pr(Y)” or “Pr(X)”. These objects do not exist. Numbers can be put in their place and the equation can be made to work out, but it is the step of putting numbers in that is wrong. There is no such thing as an unconditional probability, so we can never write without error “Pr(Y)” or “Pr(X)”. Instead, we should write e.g. Pr(Y|W) or Pr(X|W), where W is the knowledge we start with, i.e. our real prior (knowledge).
And I think at this point--that this post doesn't seem to have anything to do with Bayesian statistics at all. Like a frequentist putting uncertainty bounds on uncertainty bounds, it seems the Bayesian analogue is conditioning on conditioning.
Is this blog sound reasoning? Is there actually a distinction between "belief" and "knowledge" when specifying a prior? If a conditional density exists, such as $Pr(Y|W)$ then a marginal density exists as well $Pr(Y) = \int Pr(Y|W)Pr(W)$?