A little something that I've been wondering about for a while:

Let $P_\theta$ be a stochastically increasing (one-parameter) exponential family on the sample space $\mathcal{X}$ with $\Theta\subset\mathbb{R}$ being its natural parameter space, i.e. $\Theta$ being the set of values for which the cdf $F_\theta$ defines a probability measure. Is it always true that $$F_\theta(x)\nearrow 1\qquad\mbox{as}\qquad\theta\searrow \inf\Theta,$$ and $$F_\theta(x)\searrow 0\qquad\mbox{as}\qquad\theta\nearrow \sup\Theta$$ for all $x\in\mathcal{X}$ which are not in the boundary of $\mathcal{X}$?

Definitions, an example, speculation

A distribution $P_\theta$ over $\mathcal{X}$ parametrized by $\theta\in\Theta$ is stochastically increasing if, for fixed but arbitrary $x\in\mathcal{X}$, $F_\theta(x)$ is decreasing in $\theta$, where $F_\theta(x)=\mbox{P}_\theta(X\leq x)$.

An example is the binomial distribution, where $\mathcal{X}=\{0,1,2,\ldots,n\}$ and the natural parameter space is $\Theta=(0,1)$. $$F_\theta(x)=\sum_{k\leq x}{\binom{n}{k}}\theta^k(1-\theta)^{n-k}$$ is decreasing in $\theta$. Example:

In this setting we have $$F_\theta(x)\nearrow 1\qquad\mbox{as}\qquad\theta\searrow \inf\Theta=0,$$ and $$F_\theta(x)\searrow 0\qquad\mbox{as}\qquad\theta\nearrow \sup\Theta=1$$ for all $x$ not on the boundary of $\mathcal{X}$. At the boundary, i.e. when $x\in \{0,n\}$, only one of the limits is attained.

Note that if we restrict the parameter space to be a proper subset of $(0,1)$, such as $(0.2,0.8)$, this is no longer true: $F_\theta(5)\nearrow F_{0.2}(5)=0.174\ldots<1$ as $\theta\searrow \inf\Theta=0.2$.

This property seems to hold for all commonly used exponential families, so my guess would be that if a counterexample exists it has to be somewhat pathological. It could perhaps involve functions that cause $\exp(<\theta,T(x)>)$ to blow up for some finite $(\theta,x)$, making its integral infinite. In some sense this would make the parameter space "restricted".

A "trivial" example is perhaps the Bernoulli(p) distribution, since its sample space equals its own boundary, so that only one of the limits can be attained for each point in the sample space. But that's a wee bit boring, and I'd much rather have example where at least one point of the sample space is not in the boundary.

  • $\begingroup$ Is it possible that only discrete distributions of the one parameter exponential family are stochastically increasing? Because, say, the exponential, the chi-square, and the beta with one parameter fixed, exhibit the opposite relation between the cdf and the parameter. $\endgroup$ Apr 19, 2014 at 0:09
  • $\begingroup$ @Alecos: that would have been helpful, but is sadly not the case. Whether it increases or decreases is a matter of parametrization. In the case of the exponential distribution, for instance, it will be increasing or decreasing depending on whether one uses $E(X)$ or $1/E(X)$ as the parameter. $\endgroup$
    – MånsT
    Apr 19, 2014 at 9:39
  • $\begingroup$ If we are not restricted to natural parametrization, finding a counterexample is easier (see my answer). However, even with $\eta(\theta) = \theta$ we can go from decreasing to increasing by flipping the sign of $T(x)$. $\endgroup$ Apr 19, 2014 at 13:35
  • $\begingroup$ Note that the natural parameter for binomial is the log-odds $\log \left(\frac {p }{1-p}\right)$. But this is a monotonic increasing function of $ p $ so the result is the same. $\endgroup$ Apr 19, 2014 at 23:20
  • $\begingroup$ Juho: to be really interesting, the parametrization should be equivalent to that using the natural parameter. Perhaps my binomial example illustrates what a "valid" parametrization would be: as @probabilityislogic pointed out, I didn't use the natural parametrization in my example, but one which is a bijection of the natural parameter. $\endgroup$
    – MånsT
    Apr 20, 2014 at 18:56

2 Answers 2


If discontinuities in the density is allowed, it is possible to construct a distribution that repeats itself over two consecutive intervals, which bounds the CDF at the first interval to $[0,0.5]$ and at the second interval to $[0.5,1]$. For example, let \begin{equation} h(x) = 1, x\in [0,4] \end{equation} \begin{equation} T(x) = \left\{ \begin{array}{ll} 1 & \quad x \in [0,1) \cup[2,3) \\ 2 & \quad x \in [1,2) \cup [3,4] \end{array} \right. \end{equation} Now, for a density proportional to $h(x)e^{\theta T(x)}$, the CDF is \begin{equation} F_\theta(x) = \left\{ \begin{array}{ll} \frac{1}{2}\,\frac{x}{1+e^\theta} & \quad x \in [0,1) \\ \frac{1}{2}\, \frac{1 + (x-1)e^\theta}{1+e^\theta} & \quad x \in [1,2) \\ \frac{1}{2} + \frac{1}{2}\,\frac{x-2}{1+e^\theta} & \quad x \in [2,3) \\ \frac{1}{2} + \frac{1}{2}\, \frac{1 + (x-3)e^\theta}{1+e^\theta} & \quad x \in [3,4) \\ \end{array} \right. \end{equation} which is decreasing as a function of $\theta$ for any $x$ in the sample space, but, for example, \begin{equation} \lim_{\theta \rightarrow \infty} F_\theta(2.1) = \frac{1}{2}. \end{equation}

  • 1
    $\begingroup$ This, too, is a trick that might not be what you are looking for. Also, I wasn't sure whether it was better to edit my previous answer or post this as a separate answer. However, based on meta.stackexchange.com/questions/28471/two-answers-one-question I decided separate was better as these are two different approaches. $\endgroup$ Apr 22, 2014 at 15:54
  • $\begingroup$ I find this trick quite appealing actually! :) Thanks. $\endgroup$
    – MånsT
    Apr 23, 2014 at 6:28

Based on comments, we are not restricted to considering the natural parameter but are allowed to use the general form \begin{equation} f_\theta(x) \propto h(x) e^{T(x)\eta(\theta)} \end{equation} In this case, it is possible to 'cheat' by constructing $\eta$ so that some values in the natural space of $\eta$ are not reached with any $\theta$. For example, let \begin{equation} f_\theta(x) \propto e^{-x\,\frac{1}{1+e^\theta}},~x\geq0 \end{equation} This is a (strangely parametrized) exponential distribution with CDF \begin{equation} F_\theta(x) = 1 - e^{-x / ({1+e^\theta})}, \end{equation} which is a decreasing function for $\theta$ as desired. However, \begin{equation} \lim_{\theta\rightarrow -\infty} F_\theta(x) = 1-e^{-x} < 1. \end{equation} The case with natural parametrization $(\eta(\theta) = \theta)$ remains open (at least to me).

  • $\begingroup$ That's very clever, but not quite what I had in mind! The answer I'm looking for does not necessarily have to be written using the natural parametrization, but the parametrization should be equivalent to the natural one. Your proposal is equivalent to using the natural parametrization with a restricted parameter space, and as I wrote in my question that will indeed result in the cdf losing the proper limits. That being said, I may well reward you the bounty if not other answers turn up! $\endgroup$
    – MånsT
    Apr 20, 2014 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.