How do Bayesians compare distributions? So, I think that I have a decent grasp of the basics of frequentist probability and statistical analysis (and how badly it can be used). In a frequentist world, it makes sense to ask such a question as "is this distribution different from that distribution", because distributions are assumed to be real, objective and unchanging (for a given situation, at least), and so we can figure out how likely it is that one sample is drawn from a distribution shaped like another sample.
In the Bayesian world view, we only care about what we expect to see, given our past experiences (I'm still a bit vague on this part, but I understand the concept of Bayesian updating). If that is so, how can a Bayesian say "this set of data is different from that set of data"?
For the purposes of this question, I don't care about statistical significance, or similar, just how to quantify difference. I'm equally interested in parametric and non-parametric distributions.
 A: Think your statement through as a Frequentist and make it more specific first. A Frequentist could not say that "data set A is different from data set B", without any further clarification.
First, you'd have to state what you mean by "different". Perhaps you mean "have different mean values". Then again, you might mean "have different variances". Or perhaps something else?
Then, you'd have to state what kind of test you would use, which depends on what you believe are valid assumptions about the data. Do you assume that the data sets are both normally-distributed about some means? Or do you believe that they are both Beta-distributed? Or something else?
Now can you see that the second decision is much like the priors in Bayesian statistics? It's not just "my past experience", but is rather what I believe, and what I believe my peers will believe, are reasonable assumptions about my data. (And Bayesians can use uniform priors, which pushes things towards Frequentist calculations.)
EDIT: In response to your comment: the next step is contained in the first decision I mentioned. If you want to decide whether the means of two groups are different, you would look at the distribution of the difference of the means of the two groups to see if this distribution does or does not contain zero, at some level of confidence. Exactly how close to zero you count as zero and exactly which portion of the (posterior) distribution you use are determined by you and the level of confidence you desire.
A discussion of these ideas can be found in a paper by Kruschke, who also wrote a very readable book Doing Bayesian Data Analysis, which covers an example on pages 307-309, "Are Different Groups Equal?". (Second edition: p. 468-472.) He also has a blog posting on the subject, with some Q&A.
FURTHER EDIT: Your description of the Bayesian process is also not quite correct. Bayesians only care about what the data tells us, in light of what we knew independent of the data. (As Kruschke points out, the prior does not necessarily occur before the data. That's what the phrase implies, but it's really just our knowledge excluding some of the data.) What we knew independently of a particular set of data may be vague or specific and may be based on consensus, a model of the underlying data generation process, or may just be the results of another (not necessarily prior) experiment.
A: this paper might be of interest:
http://arxiv.org/pdf/0906.4032v1.pdf
It gives a nice summary of some frequentist and Bayesian approaches to the two sample problem, and discusses both the parametric and nonparametric cases.
It might add something to the other answers to give a simple example.  Say you have two data sets $\mathbf{x}$ and $\mathbf{y}$ where each $x_i$ and each $y_j$ is either a $0$ or a $1$.  You assume an iid Bernoulli model in both cases, so each $x_i\sim Bern(p)$ and each  $y_i\sim Bern(q)$.  Your hypothesis testing scenario in both the frequentist and Bayesian settings can be:
$\mathcal{H}_0: \: \: p=q$
$\mathcal{H}_1: \: \: p,q$ not necessarily equal.
The likelihoods for the data in each case are:
Under $\mathcal{H}_0$ : $L_0(p) = f(\mathbf{x},\mathbf{y};p) = \prod_i p^i (1-p)^{1-i} \prod_j p^j(1-p)^{1-j}$
Under $\mathcal{H}_1$ : $L_1(p,q) = f(\mathbf{x},\mathbf{y};p,q) = \prod_i p^i (1-p)^{1-i} \prod_j q^j(1-q)^{1-j}$
(since under $\mathcal{H}_0 \:\: q=p$).  A frequentist approach to the problem might be to do a Likelihood ratio test, whereby you calculate the statistic:
$W = -2\log\left\{ \frac{L_0(p_{max})}{L_1(p_{max},q_{max})}\right\},$
where $p_{max},q_{max}$ denote the maximum likelihood estimates for $p$ and $q$ under the relevant hypothesis (so $p_{max}$ in the numerator may not be the same as $p_{max}$ in the denominator).  $W$ asymptotically follows a $\chi^2_1$ distribution (see e.g. Pawitan, 2001), so you would specify a significance level and reject/fail to reject $\mathcal{H}_0$ as appropriate.
Traditionally, in the Bayesian approach the test statistic would be the Bayes factor.  First you would assume some relevant priors $p\sim \pi_0$ under $\mathcal{H}_0$ and $p,q\sim \pi_1$ under $\mathcal{H}_1$.  The Bayes factor is the ratio of marginal likelihoods, given by:
$BF = \frac{ f(\mathbf{x},\mathbf{y}|\mathcal{H}_0) }{f(\mathbf{x},\mathbf{y}|\mathcal{H}_1)} = \frac{ \int_0^1 L_0(p)\pi_0(p)dp}{\int_0^1 \int_0^1 L_1(p,q)\pi_1(p,q)dpdq}$.
The Bayes factor can be combined with some prior beliefs on the probability of $\mathcal{H}_0$ or $\mathcal{H}_1$ being true, to give the probability of $\mathcal{H}_0$ versus $\mathcal{H}_1$ after seeing the data.  If we assume apriori that each hypothesis is equally likely, so $p(\mathcal{H}_0)=p(\mathcal{H}_1) = 1/2$, then this gives:
$\frac{p(\mathcal{H}_0|\mathbf{x},\mathbf{y})}{p(\mathcal{H}_1|\mathbf{x},\mathbf{y})} = BF \times \frac{p(\mathcal{H}_0)}{p(\mathcal{H}_1)} =  BF \times \frac{1/2}{1/2} = BF.$
Intuitively, if this ratio is $>1$, then the posterior probability of $\mathcal{H}_0$ is larger than $\mathcal{H}_1$, so you would say that $\mathcal{H}_0$ has a higher probability of being true under these assumptions for the prior and model.
One nice thing about the Bayes factor is how it automatically penalises more complex models (such as $\mathcal{H}_1$ here).  A nice paper offering some more intuition is here: http://quasar.as.utexas.edu/papers/ockham.pdf.
Hope that helps along with the other answers already posted.
A: Given data, how strongly do we believe that 2 groups do not come from the same population (H_1: they do not come from the same population vs H_0: they come from the same population). This can be done with a Bayesian t-test. 
Complexity is used to figure out how much the prior is overlapping with one hypothesis. Fit is used to figure out how much the posterior is overlapping with one hypothesis. Combined you can compare the hypotheses and express your posterior belief in whether or not they come from the same population. 
