2
$\begingroup$

In the stats class I took, all the results I have encountered about the convergence in distribution of some random variables are in one way or another consequences of the Central Limit Theorem.

Out of pure curiosity, I wonder whether there exists useful and commonly used results about the convergence in distribution of some statistic $f(X_1,\dots,X_n)$ (where $(X_1,\dots,X_n)$ could be a random sample)

  1. Which do not follow from the Central limit theorem.
  2. Which holds for any distribution of the individual $X_i$, or under some "mild" conditions on their distribution.

(sorry for the "mild" I know it means nothing precise, so my appreciation of answers will be partly subjective, but I do not know of a way to formalize this. As an example, the CLT requires that $E(||X_i||^2) < \infty$, that is mild enough for me :).)

I found some interesting examples at http://www.math.uah.edu/stat/dist/Convergence.html, but appart from the CLT, they all violate condition 2 above.

Remark : I realize that "which do not follow from the Central limit theorem" might be a little vague. I do not know how to express this more precisely, but I guess I am curious about results which would not take the form :

  • $ X_n $ converges in probability to $X$.
  • $Y_n$ converges in distribution to $N(a,V)$ by the CLT.
  • So $X_nY_n$ converges in probability to $D \sim X N(a,V)$.

(e.g. Wald statistic converges in distribution to $N(0,A)'A^{-1}N(0,A) \sim \chi^2_q$)

Hope the question is clear enough, feel free to ask for clarifications.

|cite|improve this question
$\endgroup$
  • 3
    $\begingroup$ See the Fisher-Tippett-Gnedenko Theorem at en.wikipedia.org/wiki/…, for instance. Its conditions raise the question, what exactly do you mean by a distribution not "known"? $\endgroup$ – whuber♦ Apr 29 '14 at 19:18
  • $\begingroup$ Thanks for you comment. That is not clear I agree. By "not know" I meant "for any distribution". I'll edit the question. $\endgroup$ – Martin Van der Linden Apr 29 '14 at 19:26
  • $\begingroup$ @whuber : your suggestion of the Fisher-Tippett-Gnedenko Theorem is great. I know it does not hold for any random sample as the theorem relies on some conditions on the distribution of the elements of the random sample, but that fits the kind of result I was looking for. Would you consider making it an answer? $\endgroup$ – Martin Van der Linden Apr 29 '14 at 20:06
3
$\begingroup$

I used to ask myself the same question and I recently had an answer in an Extreme Value Theory course.

In this domain we focus on extreme values i.e. on $max_i(X_i)$ not the sum. (from wikipedia)

The Fisher–Tippett–Gnedenko theorem states: If there exist sequences of constants $a_n>0$ and $b_n\in \mathbb R$ such that $P((M_n-b_n)/a_n \leq z) \rightarrow G(z) \propto \exp (-(1+\zeta z)^{-1/\zeta} ) $ as $n \rightarrow \infty$ where $\zeta$ depends on the tail shape of the distribution. When normalized, ''G'' belongs to one of the following non-degenerate distribution families:

Weibull, law: $ G(z) = \begin{cases} \exp\left\{-\left( -\left( \frac{z-b}{a} \right) \right)^\alpha\right\} & z<b \\ 1 & z\geq b \end{cases}$

When the distribution of $M_n$ has a light tail with finite upper bound (or finite support). Also known as Type 3.

Gumbel,law: $G(z) = \exp\left\{-\exp\left(-\left(\frac{z-b}{a}\right)\right)\right\}\text{ for }z\in\mathbb R.$

When the distribution of $M_n$ has an exponential tail. Also known as Type 1

Frechet, law: $G(z) = \begin{cases} 0 & z\leq b \\ \exp\left\{-\left(\frac{z-b}{a}\right)^{-\alpha}\right\} & z>b. \end{cases}$ when the distribution of $M_n$ has a heavy tail (including polynomial decay). Also known as Type 2.

In all cases, $\alpha>0$.

In practice you have to find $a_n$ et $b_n$ then $\zeta$ to find the type of your initial distribution.

Some interesting results from my course:

Distrib.F: law of Max, law of Min.

Gaussian: Gumbel, Gumbel

Cauchy: Frechet, Frechet

Uniform: Weibull, Weibull

Pareto: Frechet, Weibull

Gamma: Gumbel, Weibull

Exponential: Gumbel, Weibull

If Y Weibull, log(Y) Gumbel and $Y^{-1}$ Fréchet.

The lognormal and the χ2 distributions belong to the Gumbel domain (seems both min and max)

|cite|improve this answer
$\endgroup$
1
$\begingroup$

I am not sure there is an answer to this question in the affirmative.

If you don't know the distribution of the individual X's then how can you make any statement about the distribution of some statistic of which they are a function?

Also, I don't think it is correct to say that the distribution of some some statistic follows from the Central Limit Theorem. The convergence properties arise because of the topology of the space you're working in, i.e. a sigma algebra on a Borel set. That's where the convergence properties come from, and it is because of the convergence properties that the Central Limit Theorem is valid.

Perhaps I am not understanding your question, but if you are looking for distribution-free (or nearly distribution free results), then a non-parametric textbook may hold the answer to your question.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ I think my knowledge of probability is too limited to understand your reference to the properties of the underlying sigma algebra. I guess in my understanding, the central limit theorem says that, no matter how the $X_i$ are distributed, if they are i.i.d, then $\sqrt{n} ( \sum_{i=1}^n X_i - E(X_i))$ converges in distribution to a normal $N(0, Var(X_i))$. I would call this a statement about the asymptotic distribution of the statistic of a random sample the element of which can be distributed according to any distributions. Would that be wrong? $\endgroup$ – Martin Van der Linden Apr 29 '14 at 19:33
  • 1
    $\begingroup$ Understanding the space is a matter of understanding analysis/topology concepts rather than probability. If you have taken introductory analysis or advanced calculus you know that in R, sequences converge to a number x because x exists in R. If you have "holes" in your space, like you do if you are working in Q (the rationals), then the same sequence does not converge if x is not a member of Q. This would be the case if x is an irrational number. 1/2 $\endgroup$ – rocinante Apr 29 '14 at 19:43
  • 1
    $\begingroup$ 2/2 I don't think your statement about the CLT is wrong. I am just saying that I think the CLT is an inevitable consequence of the standard space we're working in. In other words, if you can't say anything about the distribution of the X's, then I don't see how you can make any inference about a function of their distribution. You have to have some knowledge about the behavior of the X's to be able to say something about a function. The only scenario where this wouldn't be true is if you're working in some weird space. E.g it's ok for a sequence have 2 different limits. $\endgroup$ – rocinante Apr 29 '14 at 19:50
  • $\begingroup$ I see your point. It is in line with my late edits of the questions (see above). $\endgroup$ – Martin Van der Linden Apr 29 '14 at 19:54
  • 1
    $\begingroup$ Rocinante, you seem to be suggesting that the CLT is merely an assertion about the completeness of a space. It's far more than that: it says not only that certain kinds of sequences have limits, but that many different such sequences have a common limit. That's a little analogous to a result like "all Cauchy sequences of real numbers converge to zero"--and from such a point of view is equally surprising. $\endgroup$ – whuber♦ Apr 29 '14 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.