In my personal opinion I think that Bayesian methods are extremely convenient in that you need to specify only four main ingredients for solving problems of inference, prediction, and decision making.
These questions encompass almost all of the discipline of statistics: describing a data set $D$, generalizing outward inferentially from $D$, predicting new data $D^*$, and helping people make decisions in the presence of uncertainty.
Given the set $\mathcal{B}$, of propositions summarizing your background
assumptions and judgments about how the world works as far as $\theta$,
$D$ and future data $D^*$ are concerned:
(a) It's natural (and indeed you must be prepared as a Bayesian) to
specify two conditional probability distributions:
- $p(\theta|\mathcal{B})$, to quantify all information about external to $D$ that You
judge relevant; and
- $p(D|\theta,\mathcal{B})$, to quantify your predictive uncertainty, given $\theta$, about
the data set $D$ before it's arrived.
(b) Given the distributions in (a), the distribution $p(\theta|D,\mathcal{B})$ quantifi es
all relevant information about $\theta$, both internal and external to $D$,
and must be computed via Bayes's Theorem:
$$p(\theta|D,\mathcal{B})=c\times p(D|\theta,\mathcal{B})p(\theta|\mathcal{B})\hspace{.75cm}\text{(inference)}$$
where $c > 0$ is a normalizing constant chosen so that the left-hand
side of of the above equation integrates (or sums) over $\Theta$ to 1.
(c) Your predictive distribution $p(D^*|\theta,D,\mathcal{B})$ for future data $D^*$ given the observed data set $D$ must be expressible as follows:
$$p(D^*|D,\mathcal{B})=\int_{\Theta}p(D^*|\theta,D,\mathcal{B})p(\theta|D,\mathcal{B})d\theta$$
often there's no information about $D^*$ contained in $D$ if is known, in which case this expression simplifies to
$$p(D^*|\mathcal{B})=\int_{\Theta}p(D^*|\theta,D,\mathcal{B})p(\theta|D,\mathcal{B})d\theta\hspace{.75cm}\text{(prediction)}$$
(d) to make a sensible decision about which action $a$ you should take in
the face of your uncertainty about $\theta$, however, you must be prepared to specify
(i) the set $\mathcal{A}$ of feasible actions among which you're choosing, and
(ii) a utility function $U(a,\theta)$, taking values on $\mathbb{R}$ and quantifying your
judgments about the rewards (monetary or otherwise) that would
ensue if you chose action $a$ and the unknown actually took the value $\theta$ (without loss of generality you can take large values of $U(a,\theta$) to be better than small values) then the optimal decision is to choose the action $a$ that maximizes the expectation of $U(a,\theta)$ over $p(\theta|D,\mathcal{B})$, i.e.,
$$a=\arg\max_{a\in\mathcal{A}}\mathbb{E}_{(\theta|D,\mathcal{B})}U(a,\theta)=\arg\max_{a\in\mathcal{A}}\int_{\Theta}U(a,\theta)p(\theta|D,\mathcal{B})d\theta\hspace{.75cm}\text{(decision making)}$$
And thus, there is a very simple and straight forward framework for conducting statistical analysis under the Bayesian framework. Once again, all you need to specify are the following four ingredients: $p(\theta|\mathcal{B}),p(D|\theta,\mathcal{B})$,the possible actions $a$, and the utility function $U(a,\theta)$.
Now as convenient as that is, there is no complete guidance under the Bayesian framework that actually tells you how to spefficy those four ingredients and this is actually the subjective and hard part from the Bayesian point of view. The true convenience of the Bayesian framework comes from the fact that if you are prepared to specify those four ingredients, then there is a clear routine for conducting most all of statistical reasoning. Once again though this is just my opinion.
