23 reputation
3
bio website
location
age
visits member for 1 year, 2 months
seen Mar 30 '12 at 12:20
stats profile views 2

Mar
25
comment Estimate the population variance from a set of means
Thanks for clearing that up! :) The new weights you propose seems to work well.
Mar
24
comment Estimate the population variance from a set of means
I've run some simulations on this to verify the answer and I encountered a few problems. It appears that choosing $w_i=N_iN/(N-1)$ does not give the correct result (it is scaled with about $k$). Plugging this weight into your formula for $E(\hat{\sigma^2})$, gives $E[\hat{\sigma^2}]=\sigma^2(\frac{kN}{N-1}-1)$. Choosing $w_i=\frac{N_iN}{N-N_i}\frac{1}{k}$ solves this, but the resulting estimator underestimates $\sigma^2$ when the number of groups is small. I wonder if the expression for $E[\hat{\sigma}^2]$ is correct; I've tried the calculations but I have a hard time following them.
Mar
23
comment Estimate the population variance from a set of means
Thank you so much for a very thorough and in-depth answer; this completely answers my question! :)
Mar
23
awarded  Supporter
Mar
23
awarded  Scholar
Mar
23
accepted Estimate the population variance from a set of means
Mar
22
comment Estimate the population variance from a set of means
@whuber, I appreciate the answers you've given so far; it does sound like that if the number of samples used to compute each mean is identical, I could use the formula in my 2nd question ($\sigma=\sigma_{mean}\sqrt{n}$) to estimate the standard deviation of the population. If that is the case, I can easily construct my benchmarks so that each mean do have the same number of measurements, so that the case with different $N_i$s wouldn't matter.
Mar
21
comment Estimate the population variance from a set of means
@whuber, but would not multiplying by $\sqrt{n}$ account for the SD of the means being smaller? I do not propose simply using the SD of the means as an estimate for the SD of the whole population.
Mar
20
comment Estimate the population variance from a set of means
Thank you whuber! Pardon my ignorance, but would not variation within groups be reflected in their means collectively? I.e. wouldn't a large in-group variation typically result in larger variation in the means, than if the in-group variation was small, if the subsets are assumed to be independent and from the same distribution? I will also take a look at the topic of anova as well; thanks for that!
Mar
20
comment Estimate the population variance from a set of means
Yes; these are energy measurements of a CPU for an experiment I am running, and are run on the same machine under identical conditions every time. I only have the means because the granularity is too coarse to get readings on each sample, so I have to run the experiment for a number of times to get reasonable accuracy. The partitions is like this: The N1 first measurements is in partition 1, the N2 next measurements is in partition 2, and so on. Basically, what I try to get is the standard deviation for each separate e
Mar
20
awarded  Student
Mar
20
asked Estimate the population variance from a set of means