Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

Is there a simple way to determine (other than by simulation) the expected standard deviation of a sample, when sampling from a population with known distribution, for example a normal distribution with mean 25 and standard deviation 5, but the samples are included/excluded if they fall within certain limits ? In particular, I am interested in the case where the inclusion criteria is that the sampled values are between 25 and 30.

Thanks in advance.

This is not homework.

share|improve this question
Do you take a fixed sample of size $n$ from the underlying distribution and then throw away the bad observations or do you continue sampling until you get a fixed sample size of "good" observations? – cardinal Mar 30 '12 at 18:21
Thanks cardinal - the second one is correct (we are seeking a fixed sample size of observations meeting the criteria). So far I have just used the U(25,30) distribution as a guestimate, which I expect will not be far from correct, and simulations show that to be the case, but I'd like to know a more rigorous theory-based answer, if possible. – P Sellaz Mar 30 '12 at 18:32

1 Answer

up vote 1 down vote accepted

The variance of a truncated normal distribution (with untruncated mean $\mu$ and variance $\sigma^2$, and bounds $a$ and $b$) as given by Wikipedia is: $$ \sigma^2\left[1+\frac{\alpha\phi(\alpha)-\beta\phi(\beta)}{Z} -\left(\frac{\phi(\alpha)-\phi(\beta)}{Z}\right)^2\right] $$ With: $$ \alpha=\frac{a-\mu}{\sigma}, \quad \beta=\frac{b-\mu}{\sigma}, \quad Z=\Phi(\beta)-\Phi(\alpha) $$

$\phi$ and $\Phi$ are the pdf and cdf respectively of a standard normal distribution. $\phi(x)$ has analytic form $\frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}x^2}$, and $\Phi(x)$ can be computed either by numerical integration of $\phi$, or by a number of other techniques. It is usually available as a predefined function in most mathematical libraries.

share|improve this answer
Great. Thank you. – P Sellaz Mar 30 '12 at 19:50
This is the right underlying distribution and so is on the right track. The OP asks about the "expected sample standard deviation", which leaves a bit of work left to do. I suspect the answer is a bit messy due to the nonlinearity of the square-root. I think the suggestion to use a predefined function or other technique for approximating $\Phi(x)$ is generally much better than trying to do so by numerical integration, for many reasons. – cardinal Mar 30 '12 at 20:34
FWIW, I computed the value using this answer, using the parameters in my OP, and got 1.41. This compared to 1.44 for the U(25,30) approximation. – P Sellaz Mar 30 '12 at 21:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.