Understanding the concept of "Bounded in probability"

Question

My statistics book defines the concept of "bounded in probability" in the following way:

Definition 5.2.2 (Bounded in Probability). We say that the sequence of random variables $\{X_n\}$ is bounded in probability if, for all $\epsilon > 0$, there exists a constant $B_{\epsilon} > 0$ and an integer $N_{\epsilon}$ such that $$n \geq N_{\epsilon} \implies P[|X_n| \leq B_{\epsilon}] \geq 1 - \epsilon .$$

... But doesn't this mean that any sequence of R.V.'s that does not include any R.V.'s with a pdf with infinite support (i.e. $ -\infty < x < \infty $) is bounded in probability as you can always pick a higher $B_{\epsilon}$?

Answer 1

But doesn't this mean that any sequence of R.V.'s that does not include any R.V.'s with a pdf with infinite support (i.e. $−∞<x<∞$) is bounded in probability as you can always pick a higher $B_ϵ$?

The mistake you are making lies in the logic order. For a given $\epsilon$ you have to be able to choose a $B_{\epsilon}$ that works for every $\boldsymbol{X_n}$ for $\boldsymbol{\ n\geq N_{\epsilon}}$ simultaneously. You are thinking the other way around: Once you have your $X_n$ you choose your $B_{\epsilon}$. But that's not what the definition says. So, even if all your random variables have bounded support, since your are forced to first choose $B_{\epsilon}$, an $X_n$ might come along in the sequence such that, while still having bounded support, it's support is not bounded by $B_{\epsilon}$. Note that this alone doesn't necessarily stops the sequence from being bounded in probability, but it shows why your argument doen't work. As shown by Kjetil, this can indeed happen in a way that makes the sequence not bounded in probability.

Answer 2

No, let $X_n$ be distributed with the uniform distribution on $(n, n+1)$ for $n=1,2,3,\dotsc$ Then $\{X_n\}$ is a sequence of bounded random variables which is not bounded in probability.

Answer 3

We may depict the relevant distributional properties of any sequence of random variables by plotting the distribution functions of their absolute values. Such functions have values between $0$ and $1,$ growing upwards from left to right eventually to reach $1$ (if only asymptotically). This figure uses color to indicate where each distribution is in the sequence, from blue (first) through the color spectrum into the reds (far out in the sequence).

A sequence will be bounded in probability when eventually almost all the probability of the random variables lies within a bounded set. To prove such a claim, you must be ready to meet a challenge. Your opponent, a sceptic, proposes a tiny (but still positive) number $\varepsilon.$ In return, you have to furnish a bound $B_\epsilon$ and show that after removing only a finite number of variables, all the remaining ones in the sequence have at least a chance of $1-\epsilon$ of being less than $B_\epsilon$ in size.

In terms of the graphics, $\epsilon$ sets a threshold probability $1-\epsilon$ just below the upper limit of the graph:

You are looking for a horizontal position $B_\epsilon$ that blocks out a "gray" area, like so:

Everything to the right of $B_\epsilon$ is grayed out and everything below the threshold line $1-\epsilon$ is also grayed out. That leaves only the skinny white rectangle at the upper left.

Any graph that manages to rise out of the gray area, as in the yellow through red graphs here, corresponds to a random variable that achieves more than the required chance $1-\epsilon$ of being less than $B_\epsilon$ in size. The others (in blues and greens, shown eventually with dotted lines) didn't make it: even by the time they reached $B_\epsilon,$ they hadn't reached $1-\epsilon.$ These variables have too high a chance of exceeding $B_\epsilon$ in size.

To counter the sceptic, you get to vary the position of $B_\epsilon.$ As you slide it to the right, more and more of the curves have a chance of getting out of the gray area.

A sequence is bounded in probability when, no matter how skinny the sceptic might make the upper white band, you can find a finite horizontal position $B$ for which all sufficiently red curves make it out of the gray area.

If, occasionally, a red curve (that is, the distribution for a variable far out in the sequence) dips down lower than many others, you probably don't have a sequence that's bounded in probability.