0
$\begingroup$

I am seeing a strong linear relationship between log-log plot of number of samples vs std.error of the mean estimate. As far as I can tell, this is expected (e.g. link). Since std. error is calculated using sample size, I wondered if this was spurious, but the same relationship is seen with a bootstrapped estimation as well. In both cases, the slope is roughly -1/2, which is also cited in the link above.

here is an example in R:

# direct estimate of se
n <- 1e4
x <- rnorm(n, 10, 2)
df <- data.frame(nsamp = seq(100,n,100), mu = NaN, se = NaN)
for(i in seq(nrow(df))){
  xi <- x[seq(df$nsamp[i])]
  df$mu[i] <- mean(xi)
  df$se[i] <- sd(xi)/sqrt(length(xi))
}

# bootstrap estimate of se
nboot = 100
df2 <- data.frame(nsamp = seq(100,n,100), mu = NaN, se = NaN)
for(i in seq(nrow(df))){
  mus <- NaN*seq(nboot)
  for(b in seq(nboot)){
    xib <- sample(x, df2$nsamp[i], replace = TRUE)
    mus[b] <- mean(xib)
  }
  df2$mu[i] <- mean(mus)
  df2$se[i] <- sd(mus)
}

plot(se/mu ~ nsamp, df, log="xy", col=8)
points(se/mu ~ nsamp, df2)
legend("topright", legend = c("direct est.", "bootstrap est."), col=c(8,1), pch=1)

fit <- lm(log(se/mu)~log(nsamp), df)
coef(fit)[2]

fit <- lm(log(se/mu)~log(nsamp), df2)
coef(fit)[2]

enter image description here

I'm interested in a similar exploration of quantile estimation. I'm also seeing a similar relationship with an rlnorm generated dataset:

# a log-normal distribution quantile estimation
x <- rlnorm(n, meanlog = log(5), 0.5)
hist(x, n=30)
nboot = 100
df3 <- data.frame(nsamp = seq(100,n,100), mu = NaN, se = NaN)
for(i in seq(nrow(df))){
  mus <- NaN*seq(nboot)
  for(b in seq(nboot)){
    xib <- sample(x, df2$nsamp[i], replace = TRUE)
    mus[b] <- quantile(xib, 0.95)
  }
  df3$mu[i] <- mean(mus)
  df3$se[i] <- sd(mus)
}

plot(se/mu ~ nsamp, df3, log="xy", main = "95% quantile est. of rlnorm")
fit <- lm(log(se/mu)~log(nsamp), df3)
coef(fit)[2]

enter image description here

So, is this some sort of general rule? In the case of the quantile estimate, is my assumption of normality when estimating SE/mean OK?

Many thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.