I would like to simulate data based on real data captured. The real data captured is 15 observations. The simulation based on the existing data is 100 observations. I have a mean and standard deviation for the 15 observations, however how do I simulate standard deviation for a larger sample (100 observations) based on the smaller real data? Standard deviation should generally decrease with an increase in sample size, but at what rate?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ You want to be careful to distinguish between sample standard deviation, and population standard deviation. Also: welcome to CV! $\endgroup$– AlexisCommented Sep 4, 2020 at 23:47
-
5$\begingroup$ Why would standard deviation increase with a larger sample? It sounds like maybe there's an important detail missing $\endgroup$– Glen_bCommented Sep 5, 2020 at 3:06
-
$\begingroup$ @Glen_b The question says "standard deviation should decrease..." $\endgroup$– Tumaini KilimbaCommented Sep 6, 2020 at 9:05
-
$\begingroup$ Without context it's hard to tell what standard deviation it's referring to. Is it discussing the estimated standard deviation of the distribution of some statistic (like a sample mean, say?) rather than of the raw sample? $\endgroup$– Glen_bCommented Sep 6, 2020 at 10:01
-
1$\begingroup$ @Glen_b Yes, you are right, the context was missing (going through some of the answers below made me realise this). I had meant the standard deviation of a raw sample, given a smaller sample obtained from real observations and using that to simulate a larger sample. I was under the (erroneous?) impression larger raw samples have smaller standard devs but my understanding from the answers below is that I was mixing standard devs with standard error. $\endgroup$– Tumaini KilimbaCommented Sep 7, 2020 at 5:42
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
Standard error decreases as the sample size increases. Standard deviation is a related concept but perhaps not related enough to warrant such similar terminology that confuses everyone who is starting to learn statistics.
A sampling distribution is the distribution of values you would get if you repeatedly sampled from a population and calculated some statistic, say the mean, each time. The standard deviation of that sampling distribution is the standard error. For the standard error of the mean, it decreases by $\sqrt{n}$, so $s/\sqrt{n}$ as an estimate of the standard error (where $s$ is the sample standard deviation).
The standard deviation of a distribution is whatever it is, and it doesn’t care how large a sample you draw or if you even sample at all.
It sounds like you want to simulate data from a distribution with the mean and standard deviation you’ve calculated from the sample of $15$, so do that. If you’re willing to assume a normal distribution, the R command is rnorm and the Python command is numpy.random.normal.
-
1$\begingroup$ If you’re not willing to make the assumption of a normal distribution, please post a new question where you describe your problem in more detail. $\endgroup$– DaveCommented Sep 4, 2020 at 23:02
$\begingroup$
$\endgroup$
1
Standard deviation does not decrease with sample size. The bigger your sample is, the closer the standard deviation should be to the standard deviation of the population. What follows, with larger sample size the spread of the standard deviations estimated on larger vs smaller samples would decrease, because based on larger samples we would get more precise.
Below you can see a numerical example in R for this, where we simulate draws from standard normal distribution (with sd=1) for 15 and 100 samples, and then estimate standard deviations for them.
> summary(replicate(100000, sd(rnorm(15))))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.3039 0.8515 0.9762 0.9824 1.1061 1.8886
> summary(replicate(100000, sd(rnorm(100))))
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.6916 0.9498 0.9971 0.9980 1.0451 1.3089
-
$\begingroup$ Thank you for the additional clarification $\endgroup$ Commented Sep 6, 2020 at 9:14
$\begingroup$
$\endgroup$
1
You specifically ask about simulation. Following @Dave's Answer (+1), here are a couple of simulations in R.
Suppose I take a million samples of size $n = 16$ from a population distributed as $\mathsf{Gamma}(\mathrm{shape} = 4,\, \mathrm{rate}=.1),$ so that the population mean is $\mu = 40$ the population variance is $\sigma^2 = 400,$ and $\sigma = 20.$
Then the sample means (averages) $A =\bar X_{15}$ have $E(A) = 40$ and standard errors $SD(A)= \sigma/\sqrt{n} = 5.$ With a million samples, the simulation results should be accurate to about three significant digits.
set.seed(904)
a = replicate(10^6, mean(rgamma(16, 4, .1)))
mean(a); sd(a)
[1] 40.00176 # aprx 40
[1] 4.996061 # aprx 5
By contrast, let's do a similar simulation of a million samples of size $n = 100$ from the same population. Now $E(\bar X_{100}) = 40$ and $SD(\bar X_{100}) = \sigma/\sqrt{n} = 20/\sqrt{100} = 2.$
set.seed(2020)
a = replicate(10^6, mean(rgamma(100, 4, .1)))
mean(a); sd(a)
[1] 40.0014 # aprx 40
[1] 2.001084 # aprx 20/10 = 2
-
$\begingroup$ Thank you for this further illustration $\endgroup$ Commented Sep 6, 2020 at 9:12