1
$\begingroup$

Assume you draw $N$ values from a bounded set, say $[0, 1]$ without loss of generality.

If I am right, the centered values $X-m$ will remain in range $[{1\over N}-1, 1-{1\over N}]$ (by considering the most imbalanced 0/1 cases). And the standard deviation in range $[0,{1 \over 2}]$ (even $N$) or $[0,{1\over 2}\sqrt{1-{1\over N^2}}]$ (odd $N$) (by considering the most balanced 0/1 cases).

But what is the range of the centered-reduced values ${X-m\over s}$ ? How can I address this ?

$\endgroup$
5
  • 2
    $\begingroup$ I'm having trouble understanding your scenario. Do you mean $N$ samples, or one sample of size $N$? Are you assuming a probability distribution on [0,1]? And I'm not sure what you're saying about the mean being in $[1/N, (N-1)/N]$. Is this some sort of confidence interval? It doesn't seem correct though - what if $N=1$ (in which case the interval is empty)? Or $N=2$ (in which case it is the singleton $\{1/2\}$)? After that, I got really lost. $\endgroup$
    – Unwisdom
    Jan 29, 2014 at 17:19
  • $\begingroup$ I mean N values drawn from the range [0 1]. This is not a probability-related question, the distribution does not mater. I just need safety bounds to represent the centered/reduced values. Oooops, you are quite right, the mean m is obviously in range [0 1] and I wanted to express the range of X-m, which I did erroneously :( $\endgroup$ Jan 29, 2014 at 17:46
  • $\begingroup$ Since $(X-m)/s$ is invariant under changes of scale, the fact that the set is bounded is irrelevant. At this point it looks like you are pursuing Chebyshev-like inequalities, depending on what assumptions you make about the distribution of the samples. $\endgroup$
    – whuber
    Jan 29, 2014 at 17:49
  • $\begingroup$ Given $N$ arbitrary numbers $Xi$ in range $[0, 1]$, I want to know the smallest and largest values that ${Xi-m}\over s$ can reach. Either they are unbounded or they are in a finite range that should be just a function of $N$. $\endgroup$ Jan 29, 2014 at 18:00
  • 1
    $\begingroup$ Any set of $N\lt \infty$ numbers is automatically bounded between their minimum $a$ and maximum $b$. When $s$ is defined, $b-a\gt 0$. Let $z$ be any value of $(X_i-m)/s$. Then $z$ also equals $(X_i'-m')/s'$ where $X_i'=(X_i-b)/(a-b)$, $m'$ is the mean of the $X'$, and $s'$ is their SD. Notice that the $X'$ lie in the range $[0,1]$. That's why boundedness is irrelevant. $\endgroup$
    – whuber
    Jan 29, 2014 at 18:14

2 Answers 2

4
$\begingroup$

Let the data be $X_1, X_2, \ldots, X_n$ and, noting that all the $X_i$ are interchangeable, we may without loss of generality extremize $f(X_1, \ldots, X_n) = (X_1 - \bar{X})/\text{sd}(X)$ assuming $\text{sd}(X)\ne 0$ (which implies $n\gt 1$). Because $f$ is unchanged under translations and rescalings of the $X_i$, we may assume $\bar{X}=0$ and $\text{sd}(X)=1,$ whence $f(X) = X_1.$ This is a constrained linear program defined on $\mathbb{R}^n$:

Optimize

$$X_1$$

subject to

$$X_1 + X_2 + \cdots + X_n = n\bar{X} = 0$$ $$X_1^2 + X_2^2 + \cdots + X_n^2 = n(\text{sd}(X))^2 = n.$$

Introducing Lagrange multipliers $\lambda,$ $\mu,$ and $\nu$, the critical points must lie on solutions to

$$\lambda(1,0,\ldots,0)+\mu(1,1,\ldots,1)+2\nu(X_1,X_2,\ldots,X_n)=(0,0,\ldots,0).$$

It is immediate that for any solution $X_2=X_3=\cdots=X_n=-\mu/(2\nu)$. Let this common value be $Y$. Because the mean is zero,

$$X_1 = (1-n)Y.$$

Because the sum of squares of the $X_i$ is unity, we find

$$n = X_1^2 + X_2^2 + \cdots + X_n^2 = \left((1-n)Y\right)^2 + Y^2 + \cdots + Y^2 = n(n-1)Y^2,$$

yielding $Y = \pm \sqrt{1/(n-1)}$ and extreme values

$$f^{*}(X_1, X_2, \ldots, X_n) = X_1^{*} =\pm (1-n)\sqrt{1/(n-1)} = \pm\sqrt{n-1}.$$

This shows that the values of $f$ necessarily lie between $-\sqrt{n-1}$ and $\sqrt{n-1}$ and reach those extrema if and only if all but one of the $X_i$ have the same value and the remaining one has a different value. When $n\gt 2,$ $f^{*}$ can attain any value in between these extremes because $f$ is a continuous function on the connected set $\mathbb{R}^n - \{(t,t,\ldots,t)\ |\ t\in\mathbb{R}\}$. When $n=2,$ $f^{*}$ either equals $1$ or $-1$ but cannot attain any value in between.

$\endgroup$
3
  • $\begingroup$ Excellent, thanks. Imposing the value of the mean and deviation was the good idea. $\endgroup$ Jan 29, 2014 at 20:19
  • 1
    $\begingroup$ I just discovered that this result is known as Samuelson's inequality: en.wikipedia.org/wiki/Samuelson's_inequality. $\endgroup$ Feb 17, 2014 at 22:08
  • $\begingroup$ Thank you. According to Stigler's Law of Eponymy, it would be unlikely for Samuelson to be the original discoverer--and indeed, the Wikipedia article points to Laguerre a half century earlier. One tiny additional result in my answer is that all possible values between the bounds can actually be attained. $\endgroup$
    – whuber
    Feb 17, 2014 at 22:12
5
$\begingroup$

I think I understand now. You want to know what the maximum and minimum possible values of $(X-m)/s$ can be.

My feeling (I haven't worked this out analytically) is that this will be maximized when all of the observed values are the same except for one. Suppose without loss of generality that have $N-1$ observed values of 0, and one observed value of $1$. In this case, the mean will be $1/N$, the median $m$ will be $0$, and $s=1/\sqrt{N}$. The maximum value of $$\frac{X-m}{s}$$ is therefore $\sqrt{N}$.

$\endgroup$
4
  • 1
    $\begingroup$ +1 That $\sqrt{N}$ is the best possible bound follows immediately from Chebyshev's Inequality applied to the empirical CDF of the data. $\endgroup$
    – whuber
    Jan 29, 2014 at 18:15
  • $\begingroup$ Sorry, by $m$ I didn't mean the median but the arithmetic mean. $\endgroup$ Jan 29, 2014 at 19:16
  • $\begingroup$ Hmm. I was sure that I'd read the word "median" somewhere, but it must have been a different question. My mistake! $\endgroup$
    – Unwisdom
    Jan 29, 2014 at 19:19
  • $\begingroup$ Assuming $Z$ zeroes and $N-Z$ ones, the average is ${N-Z\over N}$ and the variance ${Z(N-Z)\over N^2}$; the range is [${0-m\over s}, {1-m\over s}$], i.e. [$-\sqrt{{N-Z\over Z}}, \sqrt{{Z\over N-Z}}$]. The smallest lower and largest upper bounds are obtained for $Z=1$ and $Z=N-1$ respectively, showing that the range $[-\sqrt{N-1}, \sqrt{N-1}]$ can be reached, just like you said (I am using the biaised $s$). $\endgroup$ Jan 29, 2014 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.