13
$\begingroup$

Something has been bugging me about E.T. Jaynes' treatment of continuous parameters.

In his book Probability Theory: The Logic of Science, uses notation that I am unfamiliar with when getting probabilities for continuous parameters from their densities. For example, in chapter 4, page 420, equation 4-49, Jaynes "discretizes" the continuous parameter $f \in [0,1]$ by taking the interval $(f, f+df)$ and putting a probability on it via $g$, the density of $f$:

.. so we will go back to the original probability from (4-3):

$$P(A|DX)=\frac{P(D|AX)P(A|X)}{P(D|X)}$$

Letting $A$ now stand for the proposition "The fraction of bad ones is in the range $(f, f+df)$", there is a prior pdf

$$P(A|X)=g(f|X)~df, $$

which gives the probability that the fraction of bad ones is in the range $df;$

Why isn't the above written as $P(A|X)=\displaystyle\int_f^{f+dx} g(t|X)~dt$ (with dummy variable $t$)? $g(f|X)~df$ for a finite $df$ is the left-rectangle approximation (a la Riemann sums), not $P(A|X).$ If you were to partition the $[0,1]$ into many separate proposition and assigned probability as above, it may not sum to $1.$

Or does $df$ denote some sort of infinitesimal notation I'm not familiar with? I've never seem that notation used 'alone'-- as far as I know it's only meaningful when used in context of differentiation/integration, like $\frac{d}{dx}$ or $\int \textrm{foo}()~dx$. If it is some infinitesimal notation, this seems to run contrary to Jaynes' philosophy of sticking to finite number of propositions until the end, where the limit may be taken explicitly.

As another example, in chapter 15, page 1514, equation (15-38), he writes the bivariate normal probability with correlation $\rho$ as:

$$p(dx~dy|I)=\frac{\sqrt{1-\rho^2}}{2\pi}\,\exp\left[-\frac{1}{2}(x^2 + y^2 -2 \rho x y)\right]\,dx\,dy$$

which is a little different again. Presumably $p(dx~dy|I)$ means "Probability that the true value of $(x,y)$ falls in $(x+dx,y+dy)$", but here again he writes the Riemann-sum-like approximation instead of the integral of the density on $[(x,x+dx) \times (y,y+dy)].$

What am I missing?

$\endgroup$
1

3 Answers 3

9
$\begingroup$

The $df$ should be taken as approaching zero. See differential (infinitesimal).

$\endgroup$
3
  • 1
    $\begingroup$ So should I take $g(f|X)df$ as an arbitrary close approximation to $\int_f^{f+dx} g(t|X)dt$? Shouldn't it be written as $P(A|X) \approx g(f|X)df$, since until you pass the lim of $df$ to zero at the end, as Jaynes insists, it's still positive number, right? $\endgroup$
    – biofreezer
    Commented Nov 11, 2011 at 1:36
  • $\begingroup$ Never mind, I get it now... though I think this related wiki article is much better. Thanks. $\endgroup$
    – biofreezer
    Commented Nov 11, 2011 at 2:19
  • 3
    $\begingroup$ I think this approach to integrals is archaic in that it relies on Riemann's perspective, not Lebesgue's. This is of course in tune with Jaynes' anti-measure-theory stance. $\endgroup$
    – Xi'an
    Commented Nov 11, 2011 at 6:16
10
$\begingroup$

About the first quote: Your first interpretation is correct: Jaynes replaces the integral with a rectangle approximation. Since it is an approximation to the integral, you are also correct in remarking that, for a given $df$, the finite sum over the rectangles does not give $1$.

About the second quote: I am afraid Jaynes made a notational mistake in this definition. Using the Lebesgue measure on $\mathbb{R}^2$ as $\text{d}x\,\text{d}y$, we should have $$ \text{d}p(x,y|I)=\sqrt{1−ρ^2}/\sqrt{2π}\exp[−1/2(x^2+y^2−2ρxy)]\text{d}x\text{d} y $$ for the normal measure. However, since Jaynes is overall very much opposed to measure theory, I think your interpretation as a Riemann approximation is completely right!

$\endgroup$
5
  • 5
    $\begingroup$ +1. The course on Jaynes' theory (to which you link) contains this remarkable quotation, which I think automatically settles the matter of Jaynes' credentials for any mathematician trained after the 19th century: "There are no really trustworthy standards of rigor in a mathematics that embraced the theory of infinite sets." (The course notes address some further mathematical solecisms: pp 8-10.) $\endgroup$
    – whuber
    Commented Nov 11, 2011 at 15:18
  • 1
    $\begingroup$ Update re the linked site: The current link isn't providing the relevant materials. I tried to use the archived one, but couldn't retrieve the slides. $\endgroup$ Commented Oct 11, 2022 at 8:21
  • 2
    $\begingroup$ @User1865345: Someone complained about this set of slides that quoting Jaynes was a breach of copyright (!) so I had to remove it. Here is another link. $\endgroup$
    – Xi'an
    Commented Oct 11, 2022 at 14:57
  • $\begingroup$ That's weird! It was your note. Your presentation. Thanks for the link @Xi'an. I would urge you to please add this link in your answer for the future readers for comments are prone to get deleted. As usual appreciate your contributions. +1. $\endgroup$ Commented Oct 11, 2022 at 15:18
  • $\begingroup$ Able to understand your predicament. Thanks for the edit @Xi'an! $\endgroup$ Commented Oct 11, 2022 at 21:37
6
$\begingroup$

The notation $\Pr(X \in dx)=f(x)dx$ is a standard notation to say that the law of the random variable $X$ has density $f$. Note that this notation is coherent with $\frac{\Pr(X\in dx)}{dx}=f(x)$ which actually says that $f$ is the Radon-Nikodym derivative of the law of $X$ with respect to the Lebesgue measure - in other words, the density. Thus the notation $p(dx)=f(x)dx$ is correct when denoting by $p$ the distribution of $X$. This notation is also coherent with the fact that $\Pr(X \in [x, x+h]) \approx hf(x)$ when $h$ is small.

$\endgroup$
1
  • 1
    $\begingroup$ The previous replies make it clear that although this is indeed the conventional definition, it is not the one Jaynes was using. $\endgroup$
    – whuber
    Commented Jan 8, 2012 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.