Can we report a credibility or confidence interval for a quantity measured only once, but whose distribution is obtained by Bayesian methods?

Question

Suppose you use Bayesian methods to calculate the probability density function (pdf) of a quantity of interest $X$, given its measured value $x$ (measured only once) and some other assumptions/constraints/knowledge about the experiment.

Can you then use that pdf to calculate an expected value for $X_{exp}$ (integral mean of pdf(X) * X over the whole domain) and some interval, taking e.g. $X_{left} < X_{exp} < X_{right}$ that result in a given % of the integral of the pdf?

Why the doubt? Because the people who run the experiment argue that:

1. it does not make any sense to report uncertainty for $n = 1$
2. the variability on repeats of the experiment is often much larger than the one resulting from the Bayesian pdf, so one would incur the (in their opinion absurd) scenario where uncertainty could increase as more experiments are run.
   (Note: it is true that the biological material used in the assay can behave differently depending on how it was sourced, handled, etc., so if differently behaving material is used in different runs of the experiment, the results can indeed vary a lot).

My counterargument is that we are not making any false statements if we use the pdf-derived interval for $n = 1$: we are just saying that, based on the evidence collected in this single experiment, the 'true' value $X$ for this specific batch of biological material is believed to be between such and such bounds.
If next time we run the experiment the biological material behaves very differently, resulting in a very different $X_{exp}$, IMO that is no falsification of the earlier statement, but just a manifestation of the actual non-homogeneity of the two batches of material.
It is only because we decide to treat different runs of the experiment with different biological material as measurements of the 'same' quantity, i.e. we average all $X_{exp}$'s, that we run the risk to have increased uncertainty for $n>1$ than we had for $n=1$.
This apparent contradiction is only a consequence of our own choice to aggregate data from different and maybe incompatible experiments; or am I wrong?

Besides, there is nothing stopping uncertainty from increasing on repeats of the experiments even when the measured $x$ is used, not the Bayesian-derived $X$.
So in fact I would argue that using a Bayesian method, by producing a pdf, enables one to report some rationally derived estimate of uncertainty for the (much more frequent BTW) $n = 1$ cases, whereas sticking to the measurement $x$ only allows reporting uncertainty when $n > 1$, and worse, one is then hostage of the specific batches of material that were used in the experiments, potentially resulting in far over- or under-estimated confidence intervals.

I would really appreciate to hear the view of this expert community on this topic, which is causing quite some discussion and controversy in our company.

Answer 1

You won't be able to calculate a (frequentist) confidence interval from a single observation since you cannot calculate two unknowns (mean $\bar{x}$ and variance $s^{2}$ needed for the normal approximation $\bar{x} \pm z^\ast \left(s / \sqrt{n} \right) $) with just one observation.

However, in Bayesian inference you're starting with a prior probability which is already a "full" probability distribution. Hence, to paraphrase Richard McElreath, the minimum sample size required for Bayesian inference is $n=0$.

Both with the prior $p(X)$ as well as with the posterior $p(X|x_{1})$ (where $x_{1}$ is your single data point) you will be able to calculate $X_{exp} = \int X p(X)d{X}$ or $X_{exp'} = \int X p(X|x_{1})d{X}$, respectively, or any credible interval. They will obviously be heavily influenced by your prior, but it is possible to calculate these quantities.