7
$\begingroup$

I would like to simulate data based on real data captured. The real data captured is 15 observations. The simulation based on the existing data is 100 observations. I have a mean and standard deviation for the 15 observations, however how do I simulate standard deviation for a larger sample (100 observations) based on the smaller real data? Standard deviation should generally decrease with an increase in sample size, but at what rate?

$\endgroup$
  • 2
    $\begingroup$ You want to be careful to distinguish between sample standard deviation, and population standard deviation. Also: welcome to CV! $\endgroup$ – Alexis Sep 4 at 23:47
  • 5
    $\begingroup$ Why would standard deviation increase with a larger sample? It sounds like maybe there's an important detail missing $\endgroup$ – Glen_b Sep 5 at 3:06
  • $\begingroup$ @Glen_b The question says "standard deviation should decrease..." $\endgroup$ – Tumaini Kilimba Sep 6 at 9:05
  • $\begingroup$ Without context it's hard to tell what standard deviation it's referring to. Is it discussing the estimated standard deviation of the distribution of some statistic (like a sample mean, say?) rather than of the raw sample? $\endgroup$ – Glen_b Sep 6 at 10:01
  • 1
    $\begingroup$ @Glen_b Yes, you are right, the context was missing (going through some of the answers below made me realise this). I had meant the standard deviation of a raw sample, given a smaller sample obtained from real observations and using that to simulate a larger sample. I was under the (erroneous?) impression larger raw samples have smaller standard devs but my understanding from the answers below is that I was mixing standard devs with standard error. $\endgroup$ – Tumaini Kilimba Sep 7 at 5:42
8
$\begingroup$

Standard error decreases as the sample size increases. Standard deviation is a related concept but perhaps not related enough to warrant such similar terminology that confuses everyone who is starting to learn statistics.

A sampling distribution is the distribution of values you would get if you repeatedly sampled from a population and calculated some statistic, say the mean, each time. The standard deviation of that sampling distribution is the standard error. For the standard error of the mean, it decreases by $\sqrt{n}$, so $s/\sqrt{n}$ as an estimate of the standard error (where $s$ is the sample standard deviation).

The standard deviation of a distribution is whatever it is, and it doesn’t care how large a sample you draw or if you even sample at all.

It sounds like you want to simulate data from a distribution with the mean and standard deviation you’ve calculated from the sample of $15$, so do that. If you’re willing to assume a normal distribution, the R command is rnorm and the Python command is numpy.random.normal.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ If you’re not willing to make the assumption of a normal distribution, please post a new question where you describe your problem in more detail. $\endgroup$ – Dave Sep 4 at 23:02
10
$\begingroup$

Standard deviation does not decrease with sample size. The bigger your sample is, the closer the standard deviation should be to the standard deviation of the population. What follows, with larger sample size the spread of the standard deviations estimated on larger vs smaller samples would decrease, because based on larger samples we would get more precise.

Below you can see a numerical example in R for this, where we simulate draws from standard normal distribution (with sd=1) for 15 and 100 samples, and then estimate standard deviations for them.

> summary(replicate(100000, sd(rnorm(15))))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.3039  0.8515  0.9762  0.9824  1.1061  1.8886 
> summary(replicate(100000, sd(rnorm(100))))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.6916  0.9498  0.9971  0.9980  1.0451  1.3089 
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the additional clarification $\endgroup$ – Tumaini Kilimba Sep 6 at 9:14
4
$\begingroup$

You specifically ask about simulation. Following @Dave's Answer (+1), here are a couple of simulations in R.

Suppose I take a million samples of size $n = 16$ from a population distributed as $\mathsf{Gamma}(\mathrm{shape} = 4,\, \mathrm{rate}=.1),$ so that the population mean is $\mu = 40$ the population variance is $\sigma^2 = 400,$ and $\sigma = 20.$

Then the sample means (averages) $A =\bar X_{15}$ have $E(A) = 40$ and standard errors $SD(A)= \sigma/\sqrt{n} = 5.$ With a million samples, the simulation results should be accurate to about three significant digits.

set.seed(904)
a = replicate(10^6, mean(rgamma(16, 4, .1)))
mean(a);  sd(a)
[1] 40.00176     # aprx 40
[1] 4.996061     # aprx 5

By contrast, let's do a similar simulation of a million samples of size $n = 100$ from the same population. Now $E(\bar X_{100}) = 40$ and $SD(\bar X_{100}) = \sigma/\sqrt{n} = 20/\sqrt{100} = 2.$

set.seed(2020)
a = replicate(10^6, mean(rgamma(100, 4, .1)))
mean(a);  sd(a)
[1] 40.0014     # aprx 40
[1] 2.001084    # aprx 20/10 = 2
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for this further illustration $\endgroup$ – Tumaini Kilimba Sep 6 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.