Something has been bugging me about E.T. Jaynes' treatment of continuous parameters.

In his book Probability Theory: The Logic of Science, uses notation that I am unfamiliar with when getting probabilities for continuous parameters from their densities. For example, in chapter 4, page 420, equation 4-49, Jaynes "discretizes" the continuous parameter $f \in [0,1]$ by taking the interval $(f, f+df)$ and putting a probability on it via $g$, the density of $f$:

.. so we will go back to the original probability from (4-3):


Letting A now stand for the proposition "The fraction of bad ones is in the range $(f, f+df)$", there is a prior pdf


which gives the probability that the fraction of bad ones is in the range $df$,

Why isn't the above written as $P(A|X)=\int_f^{f+dx} g(t|X)dt$ (with dummy variable $t$)? $g(f|X)df$ for a finite $df$ is the left-rectangle approximation (a la Riemann sums), not $P(A|X)$. If you were to partition the $[0,1]$ into many separate proposition and assigned probability as above, it may not sum to 1.

Or does $df$ denote some sort of infinitesimal notation I'm not familiar with? I've never seem that notation used 'alone'-- afaik it's only meaningful when used in context of differentiation/integration, like $\frac{d}{dx}$ or $\int foo()dx$. If it is some infinitesmal notation, this seems to run contrary to Jaynes' philosophy of sticking to finite number of propositions until the end, where the limit may be taken explicitely.

As another example, in chapter 15, page 1514, equation (15-38), he writes the bivariate normal probability with correlation $\rho$ as:

$p(dx\ dy|I)=\frac{\sqrt{1-\rho^2}}{2\pi}exp[-\frac{1}{2}(x^2 + y^2 -2 \rho x y)]dxdy$

Which is a little different again. Presumably $p(dx\ dy|I)$ means "Probability that the true value of $(x,y)$ falls in $(x+dx,y+dy)$", but here again he writes the Riemann-sum-like approximation instead of the integral of the density on $[(x,x+dx) \times (y,y+dy)]$.

What am I missing?


The $df$ should be taken as approaching zero. See differential (infinitesimal).

  • 1
    $\begingroup$ So should I take $g(f|X)df$ as an arbitrary close approximation to $\int_f^{f+dx} g(t|X)dt$? Shouldn't it be written as $P(A|X) \approx g(f|X)df$, since until you pass the lim of $df$ to zero at the end, as Jaynes insists, it's still positive number, right? $\endgroup$ – biofreezer Nov 11 '11 at 1:36
  • $\begingroup$ Never mind, I get it now... though I think this related wiki article is much better. Thanks. $\endgroup$ – biofreezer Nov 11 '11 at 2:19
  • 3
    $\begingroup$ I think this approach to integrals is archaic in that it relies on Riemann's perspective, not Lebesgue's. This is of course in tune with Jaynes' anti-measure-theory stance. $\endgroup$ – Xi'an Nov 11 '11 at 6:16

About the first quote: Your first interpretation is correct: Jaynes replaces the integral with a rectangle approximation. Since it is an approximation to the integral, you are also correct in remarking that, for a given $df$, the finite sum over the rectangles does not give $1$.

About the second quote: I am afraid Jaynes made a notational mistake in this definition. Using the Lebesgue measure on $\mathbb{R}^2$ as $\text{d}x\,\text{d}y$, we should have $$ \text{d}p(x,y|I)=\sqrt{1−ρ^2}/\sqrt{2π}\exp[−1/2(x^2+y^2−2ρxy)]\text{d}x\text{d} y $$ for the normal measure. However, since Jaynes is overall very much opposed to measure theory, I think your interpretation as a Riemann approximation is completely right!

  • 4
    $\begingroup$ +1. The course on Jaynes' theory (to which you link) contains this remarkable quotation, which I think automatically settles the matter of Jaynes' credentials for any mathematician trained after the 19th century: "There are no really trustworthy standards of rigor in a mathematics that embraced the theory of infinite sets." (The course notes address some further mathematical solecisms: pp 8-10.) $\endgroup$ – whuber Nov 11 '11 at 15:18

The notation $\Pr(X \in dx)=f(x)dx$ is a standard notation to say that the law of the random variable $X$ has density $f$. Note that this notation is coherent with $\frac{\Pr(X\in dx)}{dx}=f(x)$ which actually says that $f$ is the Radon-Nikodym derivative of the law of $X$ with respect to the Lebesgue measure - in other words, the density. Thus the notation $p(dx)=f(x)dx$ is correct when denoting by $p$ the distribution of $X$. This notation is also coherent with the fact that $\Pr(X \in [x, x+h]) \approx hf(x)$ when $h$ is small.

  • $\begingroup$ The previous replies make it clear that although this is indeed the conventional definition, it is not the one Jaynes was using. $\endgroup$ – whuber Jan 8 '12 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.