Suppose that we have a parameterized real-valued discrete stochastic process $x(t) :=\{x_k(t)\}_{k=1}^\infty$, such that $t$ assumes values in a compact set $T\subset \mathbb{R}^d$ for some finite positive integer $d$ (say = 1 for simplicity). Assume that for every $t\in T$, the process $x(t)$ is a second order process; i.e it has finite mean and variance functions.

My question is the following: What are the properties of the random variable defined as $\sup_{t\in T} x_k(t)$ (I am assuming that it is well-defined(?)) and the process defined as $x := \{\sup_{t\in T} x_k(t)\}_{k=1}^\infty$? Since we say that $x(t)$ is a second order process for all $t \in T$, I am wondering if this carry on to the random variables defined by taking the $\sup$, or something may go wrong?

To simplify the situation as much as possible and be able to understand the meaning of the defined process, let $x(t)$ be a nice function in $t$, say continuous over $T$. Is $x$ a second order process? i.e., do the random variables $x_k := \sup_{t\in T} x_k(t)$ have finite mean and variance functions?


1 Answer 1


Something can go wrong.

First, let's simplify a bit: the index $k$ is superfluous, so let's either drop it or focus on a fixed value of $k$ for the duration. Since all that really matters is that $T$ is compact and all paths $t\to x(t)$ are continuous, we might as well study the unit interval $T=[0,1]$ with its usual (Euclidean) topology. This guarantees the random variable $Z=\sup_{t\in T} x(t)$ is finite and is attained by at least one $t\in T.$

The question concerns whether the finite variance of every random variable $x(t)$ guarantees the finite variance of $Z.$ How could it not? A little reflection suggests a problem might occur if any small subsets $S\subset T$ could occasionally exhibit huge values of $x,$ but would do so extremely rarely, thereby keeping the variances of the $x(t)$ all finite. This suggests investigating the contrapositive: what would an infinite variance of $Z$ imply about the variances of the $x(t)$? Must they then be infinite also?

The answer is no, as I will show by constructing a counterexample. It begins by defining nice functions--say, continuous ones--whose values are universally bounded except for spikes on small subsets $S \subset T.$ We might as well make this universal bound on $T\setminus S$ equal to $0.$ Consider, then, any small interval that extends some distance $\sigma$ to either side of a number $\mu;$ namely, $S = [\mu-\sigma,\mu+\sigma].$ Let $\phi:\mathbb{R}\to[0,1]$ be a continuous decreasing function for which $\phi(0)=1$ and $\phi(1)=0.$ For instance, $\phi$ could be the survival function of any continuous random variable supported on $[0,1].$

Plot of phi

By translating and rescaling the argument $t,$ applying $\phi$ to $|t|,$ and rescaling its result by some (positive) value $Z,$ we can create a "nice" spike supported in the interval $S$ rising from $0$ to $Z.$ This spike attains the value $Z$ inside $S$ and outside of $S$ is identically zero.

Plot of a spike function

Since we're searching for a counterexample--that is, a process where the random variable $Z$ has infinite variance--let's begin with $Z$ itself: suppose it has a positively supported distribution $F(z)=\Pr(Z\le z)$ with infinite variance (which implies an infinite mean). Set $n = n(Z)$ to be the smallest integer greater than or equal to $Z$. Take a whole number $p\ge 1$ (we'll determine its value later) and partition the interval $T$ into $N=N_p(Z)=n(Z)^p$ smaller intervals

$$I_{k;N} = \left[\frac{k}{N}, \frac{k+1}{N}\right)$$

for $k=0, 1, 2, \ldots, N-1.$ (The final interval for $k=N-1$ needs to include its right endpoint at $1.$)

Now pick $k$ randomly and uniformly from its $N$ possible values, independently of $Z,$ and define the function $x$ to be the spike of height $Z$ supported on $I_{k;N}.$ Notice that any specified $t\in T$ has a chance of exactly $1/N(z)$ of being within one of these intervals.

To illustrate this construction, I drew four random variables $Y_i$ from a Cauchy distribution and set $Z_i=1+|Y_i|.$ The $Z_i$ have infinite mean, but these four particular realizations happened to be $z_i = (1.78, 2.53, 1.01, 1.18).$ Their greatest integers are $n=(2,3,2,2).$ I selected $p=3$ for this illustration, whence $N=n^3 = (8,27,8,8).$ The randomly selected values of $k$ were $k = (5,20,3,6),$ yielding these four functions in order:

Four spike functions

The spikes peak at the values $(5+1/2)/8, (20+1/2)/27, (3+1/2)/8,$ and $(6+1/2)/8,$ respectively.

I have described a stochastic process $x(t)$ having all the desired properties: by construction, every realization is a continuous real-valued function of $T,$ these realizations are governed by a definite probability distribution, and the distribution of $\sup_{t\in T}x(t)$ is $F.$ This process is determined by the distribution $F$ and an integer $p \ge 1.$

We need to consider the variance of the random variable associated with any point $t\in T;$ namely, $\operatorname{Var}(x(t)).$ To do this, let's estimate the survival function of $x(t),$ which I will call $S_p$ (to remind us that the answer will depend on $p$).

For any $z\gt 0$ and $t\in T,$ the only way $x(t)$ can possibly exceed $z$ is for $t$ to be near a spike taller than $z.$ This means $Z$ exceeds $z$ and $t$ lies in the random interval $I_{k;N(Z)}.$ (Even then it's possible (when $t$ is near the endpoints of that interval) for $x(t)$ still to be less than $z.$) This justifies the first inequality in the following approximation:

$$\eqalign{ S_p(z)&=\Pr(x(t) \gt z) \\ &\le \Pr(t \in I_{k;N_p(Z)}\text{ and } Z \gt z) \\ &= \Pr(t \in I_{k;N_p(Z)} \mid Z \gt z) \Pr(Z \gt z) \\ &\le \frac{1}{N_p(z)}\Pr(Z\gt z) \\ &\le \frac{1-F(z)}{z^p}. }$$

The second inequality follows because $N_p$ is an increasing function and the last inequality is a consequence of $N_p(z) \ge z^p.$

Since for any distribution, $1-F(z)\le 1,$ we obtain

$$S_p(z) \le z^{-p}.$$

Obviously $S_p(z)\le 1,$ too.

Since the variance is finite if and only if the (raw) second moment is finite, and an integration by parts gives

$$\mathbb{E}(x(t)^2) = 2\int_0^\infty x S_p(x) \mathrm{d}x \le 2\left(\int_0^1 x (1) \mathrm{d}x + \int_1^\infty x(x^{-p})\mathrm{d}x\right) = \frac{p}{p-2}\lt\infty$$

for all $p \gt 2,$ it follows that

For every $t\in T,$ $x(t)$ has finite variance (and therefore finite mean, too) provided $p\gt 2.$ Nevertheless, the distribution of $Z = \sup_{t\in T} x(t)$ is arbitrary and therefore can have infinite variance and even infinite mean.

To show how concrete this counterexample is, and to help the reader study it, here is R code to simulate the process $x(t).$ Of course it cannot create complete functions $x,$ but given a specified set of arguments $t_1, t_2, \ldots, t_n,$ it will compute the values $x(t_1), \ldots, x(t_n)$ for n.sim independent realizations of the process. To illustrate how the output can be analyzed, at the end it prints the average simulated value of $Z$ (the supremum process) and the average, by $t_i,$ of the simulated values $x(t_i).$ The mean of the simulated values of $Z$ will vary widely among simulations because $Z$ has infinite mean, but a typical average is around 10, while typical averages of the $x(t_i)$ are stable around 0.25.

phi <- function(t, z, mu, sigma) {
  x <- abs((t - mu) / sigma)
  ifelse(x < 1, 1-x^2*(3-2*x), 0) * z
# Create realizations of the process `x` and track them at specified points `t`.
n.sim <- 1e4
p <- 3
n <- 19                        # Number of points to track
# t <- signif((1:n-1/2)/n, 1)  # Values of 't' to track
t <- signif(sort(runif(n)), 2)
# Simulate the random variables (Z, k).
z <- 1 / runif(n.sim) # A positive random variable with infinite mean and variance
N <- ceiling(z^p)     # (In the text, ceiling(z)^p was used.)
k <- floor(runif(length(N), 0, N))
# Compute the x(t) based on the random variables (Z, k).
sigma <- 1/N
mu <- (k + 1/2)/N
x <- sapply(t, function(t) phi(t, z, mu, sigma))
# Create a data frame for plotting and further analysis.
X <- rbind(data.frame(x=c(x),
                      t=rep(t, each=n.sim),
                      iteration=rep(1:n.sim, length(t))),
mean(z)    # Will be much larger than any of the x means
sd(z)      # Will be huge
  • $\begingroup$ Thanks a lot for the answer! I think I need and will read it couple of times to follow the arguments. But I already have few questions: Are you looking at a continuous stochastic process $\{x_k(t)\}_{t\in T}$ which is indexed by $t$ (and $k$ is fixed)? I'm confused a little because I thought that (and you also stated that), for every $k$, $Z_k := \sup_{t\in T} x_k(t)$ is attained by one $t$. Doesn't this mean that $\exists t^\star_k\in T : Z_k = x_k(t^\star)$? But then $Z_k$ has to be second order because $x_k(t)$ is second order for every $k$ and all $t\in T$, $Z_k$ . What am I missing? $\endgroup$
    – user144410
    May 22, 2019 at 22:27
  • $\begingroup$ How do you conclude "But then..."? $\endgroup$
    – whuber
    May 22, 2019 at 22:28
  • $\begingroup$ "... because $x_k(t)$ is second order for every $k$ and all $t\in T$..." by assumption and the fact that the continuity in $t$ over the compact $T$ implies the equality $Z_k = x_k(t^\star_k)$ for some $t_k^\star \in T$, which means that these two random variables have the same distribution. $\endgroup$
    – user144410
    May 22, 2019 at 22:33
  • 1
    $\begingroup$ I'm afraid it doesn't imply that: that's basically the whole point. The problem is that there aren't just two random variables involved here, because $t$ itself depends on $Z$ and therefore is random. Maybe a different kind of example will help make this point. Flip a fair coin and set $x(t)=1$ for all $t$ if the coin is heads and otherwise set $x(t)=0.$ The distribution of every $x(t)$ is Bernoulli$(1/2),$ but $Z$ is identically $1,$ which obviously does not have a Bernoulli$(1/2)$ distribution. This is a counterexample to your claim that $Z$ must have the same distribution as some $x(t).$ $\endgroup$
    – whuber
    May 22, 2019 at 22:38
  • $\begingroup$ Let me write the random variables as follows: $x(t,k, \omega)$ where $\omega \in \Omega$ (the sample space). Do you mean that one should write $t_k^\star(\omega) = \arg\max_{t\in T} x(t,k, \omega)$ ? Also in this last example you gave, what is $T$? Just need to see that $x$ is continuous in $t$ over $T$. Many thanks! $\endgroup$
    – user144410
    May 22, 2019 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.