9
$\begingroup$

Assume the following set up:
Let $Z_i = \min\{k_i, X_i\}, i=1,...,n$. Also $X_i \sim U[a_i, b_i], \; a_i, b_i >0$. Moreover $k_i = ca_i + (1-c)b_i,\;\; 0<c<1$ i.e. $k_i$ is a convex combination of the boundaries of the respective supports. $c$ is common for all $i$.

I think I have the distribution of $Z_i$ right: it is a mixed distribution.
It has a continuous part, $$X_i \in [a_i, k_i), Z_i=X_i \Rightarrow \Pr(Z_i \le z_i) = \frac {z_i-a_i}{b_i-a_i}$$ and then a discontinuity and a discrete part where probability mass concentrates:
$$\Pr(Z_i=k_i) = \Pr(X_i > k_i) = 1- \Pr(X_i \le k_i)$$ $$= 1- \frac {k_i - a_i}{b_i-a_i} = 1-\frac {(1-c)(b_i-a_i)}{b_i-a_i} =c$$

So in all $$F_{Z_i}(z_i) = \begin{cases} 0\qquad z_i<a_i\\ \\ \frac {z_i-a_i}{b_i-a_i}\qquad a_i\le z_i<k_i \\ \\1\qquad k_i\le z_i\end{cases}$$

while for the mixed "discrete/continuous" mass/density function, it is $0$ outside the interval $[a_i, k_i]$, it has a continuous part that is the density of a uniform $U(a_i, b_i)$,$\frac {1}{b_i-a_i}$ but for $a_i\le z_i<k_i$, and it concentrates positive probability mass $c >0$ at $z_i = k_i$.

In all, it sums up to unity over the reals.

I would like to be able to derive, or say something about, the distribution and /or moments of the random variable $S_n \equiv \sum_{i=1}^n Z_i$, as $n\rightarrow \infty$.

Say, if the $X_i$'s are independent, it looks like $\Pr(S_n = \sum_i^nk_i) = c^n \rightarrow 0$ as $n\rightarrow \infty$. Can I "ignore" that part, even as an approximation? Then I would be left with a random variable that ranges in the interval $[\sum_{i=1}^na_i,\; \sum_{i=1}^nk_i)$, looking like the sum of censored uniforms, on their way to become "un-censored", and so maybe some central limit theorem... but I am probably diverging rather than converging here, so, any suggestions?

PS: This question is relevant, Deriving the distribution of the sum of censored variables, but @Glen_b 's answer is not what I need -I have to work this thing analytically, even using approximations. This is research, so please treat it like homework -general suggestions or references to literature are good enough.

$\endgroup$
4
  • $\begingroup$ If you need that, write the distribution of $Z_i$ as $\mu_{Z_i}(B)=P(Z_i\in B)=\int_B g(t)\,dt +c\,I_B(k_i)$, with a suitable $g$, in which $B$ is a Borel set. $\endgroup$
    – Zen
    Apr 26, 2014 at 1:11
  • $\begingroup$ @Zen I already wrote in the question that the distribution is discontinuous. Also the RHS of $f$ makes it obvious that this $f$ stands for a density in $[a_i,k_i)$, but for a probability for $k_i$ -and I do prefer compact notation. $\endgroup$ Apr 26, 2014 at 1:32
  • $\begingroup$ As far as I know, this notation with $f$ been a pdf and a pmf doesn't exist; and we do have the proper mathematical language to describe mixed distributions precisely. I doubt this notation will be accepted when you publish your research. Just my opinion, of course. You should always do it the way you like it. $\endgroup$
    – Zen
    Apr 26, 2014 at 1:39
  • $\begingroup$ @Zen Publishing is a long way ahead -and indeed, reviewers frown when they see non-established notation. This one is just a shorthand when one wants to describe a stepwise distribution in many lines. There is no "argument in favor" of it and against the established notation, like for example the one you used in a previous comment. $\endgroup$ Apr 26, 2014 at 1:52

3 Answers 3

5
$\begingroup$

I would follow Henry's tip and check Lyapunov with $\delta=1$. The fact that the distributions are mixed should not be a problem, as long as the $a_i$'s and $b_i$'s behave properly. Simulation of the particular case in which $a_i=0$, $b_i=1$, $k_i=2/3$ for each $i\geq 1$ shows that normality is ok.

xbar <- replicate(10^4, mean(pmin(runif(10^4), 2/3)))
hist((xbar - mean(xbar)) / sd(xbar), breaks = "FD", freq = FALSE)
curve(dnorm, col = "blue", lwd = 2, add = TRUE)

CLT

$\endgroup$
3
  • $\begingroup$ Indeed pretty normal. Good to know. The usual conditions for CLT were never an issue here, my question was whether there were other, perhaps subtle issues that twisted the asymptotic result and required a modified CLT. Your simulation shows that indeed the discrete discontinuity becomes negligible in probability as more variables enter the sum. $\endgroup$ Apr 26, 2014 at 2:44
  • $\begingroup$ Nothing specific, but they don't pose any problems. Think of them as well behaved finite numbers, independent of the index $i$. They may increase or decrease as $i$ grows (no specific rule), and not any one of them is disproportionately greater than the others... they represent differences in size of nevertheless "comparable" entities. So Lindeberg's condition most certainly holds $\endgroup$ Apr 26, 2014 at 3:26
  • $\begingroup$ Nice. Good luck with the next steps. Looks like an interesting problem. $\endgroup$
    – Zen
    Apr 26, 2014 at 3:33
3
$\begingroup$

Hints:

Assuming that $c$ is fixed and the $X_i$ are independent then you can calculate the mean $\mu_i$ and variance $\sigma_i^2$ of each $Z_i$: for example $\mu_i=E[ Z_i] = c\frac{a_i+k_i}{2} + (1-c)k_i$ and you know $k_i = ca_i + (1-c)b_i$.

Then, providing $a_i$ and $b_i$ do not grow too quickly, you can use the Lyapunov or Lindeberg conditions to apply the central limit theorem with the conclusion that $\displaystyle\frac{1}{\sqrt{\sum_1^n \sigma_i^2}}\left(\sum_1^n Z_i - \sum_1^n \mu_i\right)$ converges in distribution to a standard normal, or in a hand-waving sense $\sum_1^n Z_i$ is approximately normally distributed with mean $\sum_1^n \mu_i$ and variance $\sum_1^n \sigma_i^2$.

$\endgroup$
2
  • $\begingroup$ Thanks. There is no problem with the $a_i$'s and $b_i$'s, they do not grow with the index, they just fluctuate around. So you are saying essentially that the CLT can cover also random variables with mixed distributions? $\endgroup$ Apr 25, 2014 at 19:20
  • $\begingroup$ If for example the $a_i$ and $b_i$ were fixed, then you would have independent identically distributed random variables with a finite variance, so the central limit theorem would apply. Whether this is a mixture distribution or not does not affect this result. What I am saying is that you can extend this to cases where the random variables are independent but not identically distributed, provided that the means and variances stay reasonable. $\endgroup$
    – Henry
    Apr 26, 2014 at 10:24
2
$\begingroup$

My main worry in this question was whether one could apply the CLT "as usual" in the case I am examining. User @Henry asserted that one can, user @Zen showed it through a simulation. Thus encouraged, I will now prove it analytically.

What I am going to do first is to verify that this variable with the mixed distribution has a "usual" moment generating function. Denote $\mu_i$ the expected value of $Z_i$, $\sigma_i$ its standard deviation, and the centered and scaled version of $Z_i$ by $\tilde Z_i = \frac {Z_i-\mu_i}{\sigma_i}$.
Applying the change-of-variable formula we find that the continuous part is $$f_{\tilde Z}(\tilde z_i) = \sigma_if_Z(z_i) = \frac {\sigma_i}{b_i-a_i}$$ The moment generating function of $\tilde Z_i$ should be $$\tilde M_i(t) = E(e^{\tilde z_it}) = \int_{-\infty}^{\infty}e^{\tilde z_it}dF_{\tilde Z}(\tilde z_i) = \int_{\tilde a_i}^{\tilde k_i}\frac{\sigma_ie^{\tilde z_it}}{b_i-a_i}dz_i + ce^{\tilde k_it}$$

$$\Rightarrow \tilde M_i(t)=\frac {\sigma_i}{b_i-a_i}\frac{e^{\tilde k_it}-e^{\tilde a_it}}{t} +ce^{\tilde k_it}$$ with $$\tilde k_i = \frac {k_i-\mu_i}{\sigma_i},\;\; \tilde a_i = \frac {a_i-\mu_i}{\sigma_i}$$

Using primes to denote derivatives, if we have specified the moment generating function correctly then we should obtain $$\tilde M_i(0) = 1, \;\; \tilde M_i'(0) = E(\tilde Z) = 0 \Rightarrow \tilde M_i''(0) = E(\tilde Z_i^2) = \operatorname {Var}(\tilde Z_i)=1 $$ since this is a centered and scaled random variable.
And indeed, by calculating derivatives, applying L'Hopital's rule many times, (since the value of the MGF at zero must be calculated through limits), and doing algebraic manipulations, I have verified the first two equalities. The third equality proved too tiresome, but I trust that it holds.

So we have a proper MGF. If we take its 2nd-order Taylor expansion around zero, we have

$$\tilde M(t) = \tilde M(0) + \tilde M'(0)t +\frac 12\tilde M''(0)t^2 + o(t^2)$$

$$\Rightarrow \tilde M(t) = 1 + \frac 12t^2+ o(t^2)$$

This implies that the characteristic function is (here $i$ denotes the imaginary unit) $$\tilde \phi(t) = 1 + \frac 12 (it)^2 + o(t^2)= 1 - \frac 12 t^2 + o(t^2)$$.

By the properties of the characteristic function, we have that the characteristic function of $\tilde Z/\sqrt n$ is equal to

$$\tilde \phi_{\tilde Z/\sqrt n}(t)=\tilde \phi_{\tilde Z}(t/\sqrt n) = 1 - \frac {t^2}{2n} + o(t^2/n)$$

and since we have independent random variables, the characteristic function of $\frac 1{\sqrt n}\sum_i^n\tilde Z_i$ is

$$\tilde \phi_{\frac 1{\sqrt n}\sum_i^n\tilde Z_i}(t)= \prod_{i=1}^n\tilde \phi_{\tilde Z}(t/\sqrt n)=\prod_{i=1}^n\left(1 - \frac {t^2}{2n} + o(t^2/n)\right)$$

Then

$$\lim_{n\rightarrow \infty}\tilde \phi_{\frac 1{\sqrt n}\sum_i^n\tilde Z_i}(t) = \lim_{n\rightarrow \infty}\left(1 - \frac {t^2}{2n}\right)^n = e^{-t^2/2}$$

by how the number $e$ is represented. It so happens that the last term is the characteristic function of the standard normal distribution, and by Levy's continuity theorem, we have that

$$\frac 1{\sqrt n}\sum_i^n\tilde Z_i \xrightarrow{d} N(0,1)$$

which is the CLT. Note that the fact that the $Z$- variables are not-identically distributed,"disappeared" from view once we considered their centered and scaled versions and considered the 2nd-order Taylor expansion of their MGF/CHF: at that level of approximation, these functions are identical, and all differences are compacted in the remainder terms which disappear asymptotically.

The fact that idiosyncratic behavior at the individual level, from all individual elements, nevertheless vanishes when we consider the average behavior, I believe it is very well showcased using a nasty creature like a random variable having a mixed distribution.

$\endgroup$
2
  • $\begingroup$ Really cool, Alecos. My feeling is that the argument should depend on more specific conditions on the $a_i$'s and $b_i$'s. For instance: does the proof break if $(b_i-a_i)\downarrow 0$ quickly? (I know in your application this doesn't happen.) What do you think? $\endgroup$
    – Zen
    Apr 27, 2014 at 2:15
  • $\begingroup$ @Zen The issue regarding the variances of independent but non-identically distributed r.v.'s is a very subtle one, I don't think I still understand it clearly. The known Lyapunov or Lindeberg conditions are only sufficient for the CLT to hold. There are cases where the CLT holds even though these conditions do not. So I think that if we don't bound the variances, then there is no single answer, and the problem becomes totally case-specific. Even Billingsley's book is not clear on the matter. The question is what the remainder will look like, and what we can tell about it. $\endgroup$ Apr 27, 2014 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.