To illustrate what I mean please consider the following hypothetical scenario:

A person's favorite number $x\in[-1,1]$ is randomly distributed with atomless density function $f(x)$.

Furthermore, suppose that this person (after realizing what their favorite number $x$ is) calls out the absolute value of this favorite number i.e. $|x|$.

As an observer you know the structure, i.e., distribution of $x$ and the behavior of the person. Thus, after observing say $|x|=0.5$ you know that the person's favorite number is either 0.5 or -0.5.

But as a Bayesian updater what should you belief be? Does it make sense to say that you believe the persons favorite number is 0.5 with probability $$\mathbb{P}[x=0.5 \, |\, |x|=0.5]=\frac{\mathbb{P}[|x|=0.5 \,|\, x=0.5] \, f(0.5)}{f(0.5)+f(-0.5)}=\frac{ f(0.5)}{f(0.5)+f(-0.5)} ?$$

I suspect not, since any distribution is equivalent (in various senses) to changes on events of measure zero. But what should be done in such a scenario?

I would have thought such a problem would arise in economic theory (signaling games)but I have yet to find an reference dealing with this issue (any suggestions here would also be much appreciated).

  • $\begingroup$ "any distribution is equivalent (in various senses) to changes on events of measure zero" - what does this mean? $\endgroup$
    – jbowman
    Sep 14 '18 at 3:55
  • $\begingroup$ I believe that you're formula for the bayesian update is correct and I, like the other commenter, don't understand what you mean by your statement regarding equivalent distribution ? $\endgroup$
    – mlofton
    Sep 14 '18 at 4:26
  • 1
    $\begingroup$ What I mean is that in many well behaved function spaces such as $L^P$ spaces (sometimes referred to as Lebesgue space) any two functions which differ only on a set of measure zero are considered equivalent. Thus the function $f$ in my example and $g(x)=f(x)$ if $x\neq -0.5$ and $g(-0.5)=2f(-0.5)$ are considered equivalent. This however would obviously distort the probability $\mathbb{P}[x=0.5\, |\, |x|=0.5]$ if the equation I wrote is correct (which I suspect is not). $\endgroup$
    – bee
    Sep 14 '18 at 12:42
  • 1
    $\begingroup$ @bee: It's really subtle, but the two distributions will be equivalent in many aspects (i.e., CDF, mean, variance, etc.). You've just cleverly shown an aspect in which there are actually not equivalent. $\endgroup$
    – Cliff AB
    Sep 16 '18 at 4:05
  • 1
    $\begingroup$ @Xi'an: suppose $f$ and $f'$ are two pdfs for continuous distributions for RV's $X$, $X'$, with $f$ and $f'$ equal up to a countable set of points. Then the CDF's for $X$ and $X'$ are equal, which implies so will the mean, variance, etc. Yet the inference we make based on the observation of elements in the countable set of disagreement between the two pdfs is not. Of course, as you point out in your answer, the measure of this set is 0, so you could argue this is of no real consequence. $\endgroup$
    – Cliff AB
    Sep 16 '18 at 20:49

The paradox is one of measure theory and conditioning rather than one of Bayesian inference (and thus you should modify the title of the question). To quote Andrei Kolmogorov,

"The concept of a conditional probability with regard to an isolated hypothesis whose probability equals 0 is inadmissible."

When one defines the density $f$ of the random variable $X$, it can indeed be anything including the null function on any set $A\subset(-1,1)$ of measure zero. However, it seems to me that the easiest explanation is that one cannot choose the set $A$ a posteriori, that is once $X$ or $|X|$ is observed to be $x$, so that $x\in A$. Meaning that the actual observation $x$ (or more precisely the actual realisation $x$ of the random variable $X$) has probability zero to belong to $A$.

When setting $$\mathbb{P}[X=0.5 \, |\, |X|=0.5]=\frac{\mathbb{P}[|X|=0.5 \,|\, X=0.5] \, f(0.5)}{f(0.5)+f(-0.5)}=\frac{ f(0.5)}{f(0.5)+f(-0.5)}$$ (a) the first equality is an incorrect application of Bayes' formula for sets, since the sets are of measure zero and (b) conditioning on the set of measure zero $\{\omega;|X(\omega)|=0.5\}$ is to be understood, rather than as a conditional probability, as setting the value of the function$$\mathbb{E}[\mathbb{I}_{X=|X|}||X|=x]$$ at $x=0.5$, which is not uniquely defined since the only constraint is the definition of conditional expectations as $$\mathbb{P}[X=|X|]=\mathbb{E}\{\mathbb{E}[\mathbb{I}_{X=|X|}||X|]\}$$

  • $\begingroup$ Thanks for the response! Do you agree that the issue more generally is about moving between an atomless measure (given by $f$) to a discrete measure (the posterior $\mathbb{P}[X=x\, |\, |X|=0.5]$) ? In the sense that, $f$ is absolutely continuous w.r.t. Lebesgue measure but $\mathbb{P}[X=x\, |\, |X|=0.5]$ is not. Thus, the translation between measures can never be invariant to measure zero events and in particular Radon-Nikodym theorem isn't applicable. This is along the lines of what @CliffAB was suggesting, I believe. $\endgroup$
    – bee
    Sep 16 '18 at 15:33
  • $\begingroup$ No, I do not see this as an explanation. The same could be said of any conditional distribution of a discrete variable conditional on an absolutely continuous one. And I do not get the connection with the Radon-Nikodym theorem, I am afraid. $\endgroup$
    – Xi'an
    Sep 16 '18 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.