Definition (Consistency)

Let $T_1,T_2,\cdots,T_{n},\cdots$ be a sequence of estimators for the parameter $g(\theta)$ where $T_{n}=T_{n}(X_1,X_2,\cdots,X_{n})$ is a function of $X_{1},X_{2},\cdots,X_{n}.$

The sequence $T_{n}$ is a weakly consistent sequence of estimators for $\theta$ if for every $\varepsilon>0,$ $$\lim_{n\rightarrow\infty}P_{\theta}(|T_{n}-g(\theta)|<\varepsilon)=1.$$ If $T_{n}$ converges with probability one or almost surely (a.s.) to $g(\theta)$, that is, for every $\theta\in\Theta$ $$P_{\theta}\left(\lim_{n\rightarrow\infty}T_{n}=g(\theta)\right)=1,$$ then it is strongly consistent.

Strongly consistency implies weakly consistency.This definition says that as the sample size $n$ increases,the probability that $T_{n}$ is getting closer to $\theta$ is approaching $1$.

I am confused about what is the ${\color{Red}{\left(\Omega,\mathcal{F},P_{\theta}\right)}}$ those $T_{n},\,n=1,2,\ldots$, defined on? What's the specific probability measure ${\color{Red} {P_{\theta}}}$ ?

  • 2
    $\begingroup$ The short answer is that the triple $(\Omega, P, \mathcal{F})$ is irrelevant since, for a given problem, you can always provide one if needed. $\endgroup$
    – utobi
    Commented Apr 12, 2023 at 7:05

3 Answers 3


I deem a generalized framework formalizing the concepts at work is apt here. For more details, refer to $\rm [I].$

Let $(\Omega, \boldsymbol{\mathfrak A}, \Pr)$ be a probability space. Consider the sequence of probability spaces $\langle (\mathcal X_i, \boldsymbol{\mathfrak A}_i, \mathbf P_i)\rangle_{i=1}^\infty,$ where $(\mathcal X_i, \Vert \cdot \Vert_i)$ is a normed linear space.

Consider a sequence of rvs $\langle X_i\rangle_{i=1}^\infty$ and a sequence of real numbers $\langle r_i\rangle_{i=1}^\infty.$ Then $X_n = o_P(r_n)\iff \Pr[\Vert X_n \Vert_n\leq c|r_n|] = 1, ~\forall c >0.$

Now consider a sequence of measurable functions $f_n:\mathcal X_n\to \mathcal R ,~\mathcal R$ being a normed linear space with Borel $\sigma$-field. Define $T_n := f_n(X_n)$ and $T: \Omega \to \mathcal R. $ Then $T_n$ converges in probability to $T$ if and only if $\Vert T_n - T\Vert = o_P(1).$

Now consider a parametric family of distributions $\{\mathbf P_\theta\mid \theta\in \Theta \}$ on a sequence space $\mathcal X^\infty.$ Define a measurable function $g: \Theta\to \mathcal G, ~\mathcal G$ being a metric space with Borel $\sigma$-field. Take $\mathcal X_n = \mathcal X^n$ and take measurable functions $T_n: \mathcal X_n\to \mathcal G.$ Then $T_n$ is consistent for $g(\theta)$ if for each $\theta, ~T_n\overset{\mathbf P}{\to} g(\theta).$

The simplest and most common instance is taking $\Omega = \mathbb R^\infty, ~\mathcal X_n = \mathbb R^n.$

Observe how the underlying probability space is at work here based on the implications of the characterization of the convergence in probability above.

(Also, as a footnote, one can see how in probability can be generalized: Take $S\subseteq \prod_{i=1}^\infty \mathcal X_i.$ Then $S$ occurs in probability (denoted by $\mathcal P(S)$) if, for each $i,$ there exists $S_i(\varepsilon)\in\boldsymbol{\mathfrak A}_i$ such that $\prod_{i=1}^\infty S_i(\varepsilon)\subseteq S$ and for each $\varepsilon > 0,~\mathbf P_i(S_i(\varepsilon))\geq 1-\varepsilon. $ To see how powerful it is, consider $f_n:\mathcal X_n \to \mathbb R$ and, as above, take $T_n = f_n(X_n).$ Now, define $S:= \left\{\langle x_i\rangle_{i=1}^\infty\mid\lim_{n\to\infty} f_n(x_n) = 0\right\}.$ Then $T_n = o_{\mathbf P}(1)\iff \mathcal P(S).$)


$\rm [I]$ Theory of Statistics, Mark J. Schervish, Springer-Verlag, $1995,$ sec. $7.1.2,$ pp. $395-398.$

  • 1
    $\begingroup$ super complete answer as usual; +1 to you! $\endgroup$
    – utobi
    Commented Apr 18, 2023 at 7:10

It is customary in probability or mathematical statistics to encounter statements such as

Let $X$ be an absolutely continuous random variable with density $f$

with no reference to underlying probability space. However, we can always supply an appropriate space as follows.

Take $\Omega = \mathbb{R}$, $\mathcal{F} =$ Borel sets, $P(B) = \int_B f(x)\,dx$ for all $B\in \mathcal{F}$. If $X(w) = \omega$, $\omega \in\Omega$, then $X$ is absolutely continuous and has density $f$.

In a sense, it does not make any difference how we arrive at $\Omega$ and $P$; we may equally use a different $\Omega$ and different $P$ and a different $X$, as long as $X$ is absolutely continuous with density $f$. No matter what construction we use, we get the same essential result, that is

$$ P(X\in B) = \int_B f(x)\,dx. $$

Therefore, questions about probabilities of events involving $X$ are answered completely by knowledge of the density $f$. This implies that probabilities of events of $T(X_1,\ldots,X_n)$ are also defined by the density of $T$.

  • $\begingroup$ I agree with the basic point; unless the axiom of choice was used there's always a way to set it up, but plenty of common random variables are not continuous, for example binomial ones, also please make it clear that $X = id_\Omega$. Also why use the Borel $\sigma$-algebra instead of the Lebesgue? $\endgroup$ Commented Apr 12, 2023 at 9:35
  • $\begingroup$ @LukasLohse the substance is the same. For discrete r.v. you can choose a suitable $\Omega$, $\mathcal F$ as the sets of all subsets and the counting measure. $\endgroup$
    – utobi
    Commented Apr 12, 2023 at 10:40
  • $\begingroup$ What exactly do you mean with $X$ "having" a density $f$? I thought RVs were functions from the sample space to, say, the real numbers. As far as I understand, a PDF of random variable $X$ is a function s.t. $ \mathbb{P}(a < X < b) = \int_a^b f_X(x) dx $ -- but this would presuppose a probability measure $\mathbb{P}$! I'm confused. $\endgroup$
    – ngmir
    Commented Apr 13, 2023 at 15:57

Let's start by setting up each individual $X_i$ as a function $X_i: \Omega_i \to S_i$, with $S_i$ being a set and $\mathcal{F}_i$ and $P_{i, \theta}$ defined appropriately. Now $T_n = T_n(X_1, X_2, ..., X_n)$ is a small abuse of notation as random variables are functions from $\Omega$, while the right hand side is a "deterministic" function $t_n$ from $S_1\times S_2\times ...\times S_n$. So I would rewrite $$ T_n := t_n(X_1, X_2, ..., X_n) $$ with $ \omega_i \in \Omega_i $ and with this we can consider $$ t_n(X_1(\omega_1), X_2(\omega_2),..., X_n(\omega_n)) = T_n(\omega_1, \omega_2, ..., \omega_n)$$ which tells us that $T_n$ is a function from $\Omega_1\times \Omega_2\times ...\times \Omega_n$, which therefore can be used to define $\Omega$. Now $\mathcal{F}$ and $P_\theta$ can be anything as long as $\mathcal{F}_i$ and $P_{i, \theta}$ are their projections down to the individual $\Omega_i$.

To answer the questions in your comments I will be a little bit more explicit: If $X_i$ are iid with $(\Omega_x, P_{x, \theta}, \mathcal{F}_x)$, then $\mathcal{F}$ is actually not $\mathcal{F}_x^n$ but instead the $\sigma$-Alegabra generated by $\mathcal{F}_x^n$ through intersection, compliments and $\sigma$-countable unions. $P_\theta$ is a probability measure that means a function from $\mathcal{F}$ to $[0, 1]$ with certain properties. Now $A \in \mathcal{F}_x^n$ then $A = A_1 \times ... \times A_n, A_i \in \mathcal{F}_x$ and $P_\theta(A) = P_{x, \theta}(A_1) \cdot ...\cdot P_{x, \theta}(A_n)$. If $A\in\mathcal{F}/\mathcal{F}_x^n $ then $P_{\theta}(A)$ is determined by how it was generated from the elements of $\mathcal{F}_x^n$

When you write $P_\theta(|T_n - \theta| < \varepsilon)$ it really means $P_\theta(\{\omega \in \Omega: |T_n(\omega) - \theta| < \varepsilon\})$, with $\{\omega \in \Omega: |T_n(\omega) - \theta| < \varepsilon\} \in \mathcal{F} $


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.