Let $X_1, \dots, X_n$ be independent normally distributed random variables. What is the distribution of: $$ Y_i = \frac{X_i}{\mathrm{stdDev}(X_1, \dots, X_n)}, $$ where $\mathrm{stdDev}(X_1, \dots, X_n)$ is the standard deviation of the sample? I came across this in a simulation, where the simulated random variables were "normalised" before being used, but no statistical analysis was provided.
$\begingroup$
$\endgroup$
8
-
$\begingroup$ There are at least 2 common definitions of sample variance. You should specify the functional form you are using for sample variance. $\endgroup$– wolfiesCommented Feb 9, 2015 at 9:52
-
$\begingroup$ @wolfies, you mean if it is divided by $n$ or $n-1$? I seriously can't remember, but I think I am happy to work with the one that secures easier treatment. $\endgroup$– GrzenioCommented Feb 9, 2015 at 10:51
-
1$\begingroup$ Dividing by a variance usually would not be considered a form of "normalization" at all. Are you sure you didn't need to divide by the square root of the variance? $\endgroup$– whuber ♦Commented Feb 10, 2015 at 16:18
-
$\begingroup$ @whuber, of course you are correct. Let me fix the question $\endgroup$– GrzenioCommented Feb 10, 2015 at 17:31
-
$\begingroup$ Did you also perhaps subtract the mean of the $X_j$ from each $X_i$, producing $(X_i - \bar X)/\text{SD}(X_1,\ldots, X_n)$? This would have standardized the variables in the usual way. $\endgroup$– whuber ♦Commented Feb 10, 2015 at 19:36
|
Show 3 more comments
1 Answer
$\begingroup$
$\endgroup$
5
Not intended as an answer ... but more a comment that is too long for the comment box ...
Updated for OP's change of sample variance to sample standard deviation
To get an idea of the difficulty of the problem ... consider the simplest possible form this question can take, namely:
- a sample of size $n = 2$, where ...
- $X_1$ and $X_2$ are random draws from a common standard Normal parent.
Then, using the $(n-1)$ version of sample variance, and defining sample standard deviation as the square root of the latter, ... the problem is to find the distribution of:
$$Y = \frac{\sqrt{2}\, X_1}{\big|X_1-X_2\big|} \quad \quad \text{where } X_i \sim N(0,1) $$
This does not appear to be easy at all ... never mind solving for general $n \geq2$.
Monte Carlo simulation of the pdf (for different sample sizes $n$)
What will the general $n$ solution look like? The following diagram constructs the empirical Monte Carlo pdf of your ratio $Y$, for samples of size $n = 2, 3, 5$ and $25$. Each plot compares the:
- empirical Monte Carlo pdf [squiggly blue curve] to
- a standard Normal pdf (dashed red curve)
There is perhaps some solace in that, by the time $n = 25$, the distribution appears to be well-approximated by a standard Normal (provided the parent is standard Normal). But the small sample sizes are tricky.
- If you are interested in general Normal distributions (i.e. with non-zero means), then there is, of course, the additional complication of asymmetry to the plots.
-
$\begingroup$ Thanks for the hard work. The bimodal shape in the cases with small $n$ is really surprising, are you sure it is correct? And luckily I am only interested in zero mean case. $\endgroup$– GrzenioCommented Feb 10, 2015 at 9:57
-
$\begingroup$ Hi, @whuber correctly pointed out that I screwed up the question, I am really sorry about that. $\endgroup$– GrzenioCommented Feb 11, 2015 at 8:48
-
$\begingroup$ Not a problem - easy to update to the new form. And yes - it is correct. You can easily generate some sample Monte Carlo data yourself. In Mathematica, for instance, for n=3: ..............
data = Table[yy = RandomReal[NormalDistribution[0, 1], 3]; yy[[1]]/StandardDeviation[yy], {10^6}];... and then plot a histogram ofdata$\endgroup$– wolfiesCommented Feb 11, 2015 at 13:10 -
$\begingroup$ Shouldn't it be $\sqrt{X_1^2 + X_2^2}$ in the denominator instead of $|X_1-X_2|$? $\endgroup$– GrzenioCommented Feb 12, 2015 at 8:48
-
$\begingroup$ Nope. Sample variance is $$\frac{1}{n-1}\sum _{i=1}^n \left(x_i-\frac{\sum _{i=1}^n x_i}{n}\right){}^2$$. Sample standard deviation is the square root of that. Throw in $n=2$, and do the sums yields $$\frac{\left| X_1-X_2\right| }{\sqrt{2}}$$ $\endgroup$– wolfiesCommented Feb 12, 2015 at 12:14