I've been a statistician for a long time and have recently moved towards more information theoretic research. Because of this, the question of epistemic uncertainty in classical probability has been on my mind, particularly how to properly account for the separation of epistemic and aleatory uncertainty in simple probabilistic models.

For example, if we consider the random variable associated with a coin toss $X$, according to frequentist interpretations there is some objective $P(X=1) = p_f$ associated with the long run frequency of the outcome. This makes sense if the coin toss (or any other experiment) could truly be repeated in a completely identical way, but most physical processes are not time-invariant in such a way (or simply can't be repeated, like a particular sports match). From a Bayesian perspective $P(X=1) = p_b$ is not an objective quantity at all, but a degree of belief that may change with more or less information about the outcome.

My question is this:

For events that can not be infinitely repeated in an identical way, is there a way to account for both the event having some truly objective probability $p$ and having some transitional degree of belief under imperfect information? Heavily abusing notation, something like $\lim_{i \to \inf} P(X=1 | i) = p$

I've been a statistician for a long time and have recently moved towards more information theoretic research. Because of this, the question of epistemic uncertainty in classical probability has been on my mind, particularly how to properly account for the separation of epistemic and aleatory uncertainty in simple probabilistic models.

For example, if we consider the random variable associated with a coin toss $X$, according to frequentist interpretations there is some objective $P(X=1) = p_f$ associated with the long run frequency of the outcome. This makes sense if the coin toss (or any other experiment) could truly be repeated in a completely identical way, but most physical processes are not time-invariant in such a way (or simply can't be repeated, like a particular sports match). From a Bayesian perspective $P(X=1) = p_b$ is not an objective quantity at all, but a degree of belief that may change with more or less information about the experiment.

My question is this:

For events that can not be infinitely repeated in an identical way, is there a way to account for both the event having some truly objective probability $p$ and having some transitional degree of belief under imperfect information $i$? Heavily abusing notation, something like $\lim_{i \to \inf} P(X=1 | i) = p$