How many standard deviations around the mean do I need to guarantee covering at least one sample I am sampling readings $X = \{ x \}_i$, $i=1..N$ from an unknown distribution. I am estimating the mean $\mu$ and the standard deviation $\sigma$ from the sample.
My question is: how many $\sigma$-s do I need so that I am guaranteed that, given any distribution independently of how "pathological" it is, I will always have at least one sample within $[ \mu - n \sigma; \mu + n \sigma ]$ ?
My "intuition" tells me that the worst distribution I can have is the one where "all samples are furthest from the mean at the same time", ie for example $X = \{ -1.0, 1.0 \}$, in which case $\mu = 0$, $\sigma = 1$, so I would guess that $n=1$ would be the value I am looking for. But is this correct?
 A: You should use different notation for the sample mean and standard deviation, for example $\bar x$ and $s$.  There are no guarantees for the location of sample observations compared to the population mean and standard deviation, just probabilities.
There is a slight issue if your calculation of the sample standard deviation $s$ uses the $\frac1{n-1}$ expression of $s= \sqrt{\frac{1}{n-1}\sum(x_i-\bar x)^2}$ rather than $s= \sqrt{\frac{1}{n}\sum(x_i-\bar x)^2}$, as for $n=1$ the $\frac1{n-1}$ expression will give $\frac00$.  When $n=1$, you have the observation equal to the sample mean.
Ignoring that issue and using either expression for $s$, you can say that for all $n>0$, at least one sample observations will be in the interval $[\bar x -s, \bar x +s]$. If none were then you would have  $\sum(x_i-\bar x)^2 > \sum_i s^2  = n s^2$, leading to the contradictory $s>s$.
All the sample observations will be in the interval $[\bar x -\sqrt{n}s, \bar x +\sqrt{n}s]$.  If any observation $x_j$ was outside that interval you would have $\sum(x_i-\bar x)^2 >  (x_j-\bar x)^2 > ns^2$ again leading to the contradictory $s>s$.
Both $[\bar x -s, \bar x +s]$ and $[\bar x -\sqrt{n}s, \bar x +\sqrt{n}s]$ are narrower than $[\bar x -ns, \bar x +ns]$ for $n>0$, with equality only when $n=1$.
