Let $\theta > 0$ be some model parameter for which properties (bias, ...) of an estimate are studied via simulations.

For a given data set, an estimate $\hat{\theta}_i$ of $\theta$ can be obtained by maximising a likelihood function.

As $\theta$ is positive, however, I perform the maximisation over $\eta = \log(\theta) \in \mathbb{R}$ and I set $\hat{\theta} = \exp(\hat{\eta})$.

After running $N$ simulations, how should I compute the mean value,

• $\bar{\hat{\theta}} = \frac{1}{N} \sum_{i=1}^N \hat{\theta}_i$, or
• $\bar{\hat{\theta}} = \exp(\frac{1}{N} \sum_{i=1}^N \hat{\eta}_i)$?

Example:

-- Minus log likelihood for a sample from the Exp distribution with mean $\theta = \exp(\eta)$:

minusloglik <- function(eta, sample)
{
  theta <- exp(eta)
  - sum(dexp(x=sample, rate=theta, log=TRUE))
}

-- True value of $\theta$:

theta <- 5.73

-- Simulations:

thetaHat <- etaHat <- rep(NA, 1000)
for(i in 1:1000)
{  
  sample <- rexp(n=100, rate=theta)
  etaHat[i] <- nlm(f=minusloglik, p=0, sample=sample)$estimate
  thetaHat[i] <- exp(etaHat[i])
}

Question:

• Should I summarise the results as mean(thetaHat) or as exp(mean(etaHat))?
• Is the answer the same if $\theta$ denotes the variance of a normal distribution?