6
$\begingroup$

Let $X_1, \dots, X_n$ be independent normally distributed random variables. What is the distribution of: $$ Y_i = \frac{X_i}{\mathrm{stdDev}(X_1, \dots, X_n)}, $$ where $\mathrm{stdDev}(X_1, \dots, X_n)$ is the standard deviation of the sample? I came across this in a simulation, where the simulated random variables were "normalised" before being used, but no statistical analysis was provided.

$\endgroup$
8
  • $\begingroup$ There are at least 2 common definitions of sample variance. You should specify the functional form you are using for sample variance. $\endgroup$
    – wolfies
    Feb 9, 2015 at 9:52
  • $\begingroup$ @wolfies, you mean if it is divided by $n$ or $n-1$? I seriously can't remember, but I think I am happy to work with the one that secures easier treatment. $\endgroup$
    – Grzenio
    Feb 9, 2015 at 10:51
  • 1
    $\begingroup$ Dividing by a variance usually would not be considered a form of "normalization" at all. Are you sure you didn't need to divide by the square root of the variance? $\endgroup$
    – whuber
    Feb 10, 2015 at 16:18
  • $\begingroup$ @whuber, of course you are correct. Let me fix the question $\endgroup$
    – Grzenio
    Feb 10, 2015 at 17:31
  • $\begingroup$ Did you also perhaps subtract the mean of the $X_j$ from each $X_i$, producing $(X_i - \bar X)/\text{SD}(X_1,\ldots, X_n)$? This would have standardized the variables in the usual way. $\endgroup$
    – whuber
    Feb 10, 2015 at 19:36

1 Answer 1

2
$\begingroup$

Not intended as an answer ... but more a comment that is too long for the comment box ...

Updated for OP's change of sample variance to sample standard deviation

To get an idea of the difficulty of the problem ... consider the simplest possible form this question can take, namely:

  • a sample of size $n = 2$, where ...
  • $X_1$ and $X_2$ are random draws from a common standard Normal parent.

Then, using the $(n-1)$ version of sample variance, and defining sample standard deviation as the square root of the latter, ... the problem is to find the distribution of:

$$Y = \frac{\sqrt{2}\, X_1}{\big|X_1-X_2\big|} \quad \quad \text{where } X_i \sim N(0,1) $$

This does not appear to be easy at all ... never mind solving for general $n \geq2$.

Monte Carlo simulation of the pdf (for different sample sizes $n$)

What will the general $n$ solution look like? The following diagram constructs the empirical Monte Carlo pdf of your ratio $Y$, for samples of size $n = 2, 3, 5$ and $25$. Each plot compares the:

  • empirical Monte Carlo pdf [squiggly blue curve] to
  • a standard Normal pdf (dashed red curve)

enter image description here

There is perhaps some solace in that, by the time $n = 25$, the distribution appears to be well-approximated by a standard Normal (provided the parent is standard Normal). But the small sample sizes are tricky.

  • If you are interested in general Normal distributions (i.e. with non-zero means), then there is, of course, the additional complication of asymmetry to the plots.
$\endgroup$
5
  • $\begingroup$ Thanks for the hard work. The bimodal shape in the cases with small $n$ is really surprising, are you sure it is correct? And luckily I am only interested in zero mean case. $\endgroup$
    – Grzenio
    Feb 10, 2015 at 9:57
  • $\begingroup$ Hi, @whuber correctly pointed out that I screwed up the question, I am really sorry about that. $\endgroup$
    – Grzenio
    Feb 11, 2015 at 8:48
  • $\begingroup$ Not a problem - easy to update to the new form. And yes - it is correct. You can easily generate some sample Monte Carlo data yourself. In Mathematica, for instance, for n=3: .............. data = Table[yy = RandomReal[NormalDistribution[0, 1], 3]; yy[[1]]/StandardDeviation[yy], {10^6}]; ... and then plot a histogram of data $\endgroup$
    – wolfies
    Feb 11, 2015 at 13:10
  • $\begingroup$ Shouldn't it be $\sqrt{X_1^2 + X_2^2}$ in the denominator instead of $|X_1-X_2|$? $\endgroup$
    – Grzenio
    Feb 12, 2015 at 8:48
  • $\begingroup$ Nope. Sample variance is $$\frac{1}{n-1}\sum _{i=1}^n \left(x_i-\frac{\sum _{i=1}^n x_i}{n}\right){}^2$$. Sample standard deviation is the square root of that. Throw in $n=2$, and do the sums yields $$\frac{\left| X_1-X_2\right| }{\sqrt{2}}$$ $\endgroup$
    – wolfies
    Feb 12, 2015 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.