7
$\begingroup$

I have a vague feeling that for a fixed sample size, lower-order moments of a distribution would typically be estimated more precisely than higher-order moments. E.g. mean would be estimated more precisely than the second moment.

  1. How do I phrase this formally?*
  2. Is this correct (perhaps under certain conditions)? What are the conditions for this to hold?

The question is motivated by the following thread at Quantitative Finance Stack Exchange: "Why is asset volatility easier to estimate than the asset mean if it contains the mean?"

*There are different measures of precision and I wonder which one(s) may make most sense; perhaps there is a standard way of thinking about this problem that I am not aware of.

Improve this question
$\endgroup$
13
  • $\begingroup$ asking for measures of precision while saying that there are different ones without furnishing the post with some examples might distract the average responder from the real question at hand: moment estimation $\endgroup$
    – develarist
    Nov 12, 2020 at 11:22
  • $\begingroup$ Also it's irrelevant if you think that lower order moments are easier to estimate. The real question might actually be whether even vs odd moments of financial data are easier to estimate. Also, some people don't automatically recognize that mean and variance are "lower-order moments", so it might have to be spelled out for them in order to avoid drawing blanks $\endgroup$
    – develarist
    Nov 12, 2020 at 11:26
  • 2
    $\begingroup$ @develarist, thanks, I will see if I can incorporate your suggestions in the post. The question is about lower- vs. higher-order moments, not even vs. odd; that would be a separate question. The real-world interpretation of the random variables is immaterial for the question. Variance is a central moment, so its estimation precision is a bit tangential to the question. $\endgroup$
    – Richard Hardy
    Nov 12, 2020 at 11:36
  • 1
    $\begingroup$ If you look into rates of convergence, most estimators for higher moments converge slower than estimates for lower order moments. However, there is one area where this is not true: financial return forecasting. Forecasting returns is difficult simply because if it were easy, many people would do it and that would quickly eat up the forecasted return leaving no more for others. This is one of the few cases where a moment is the result of an equilibrium which perturbs the moment. So, apart from that problem domain, look into asymptotics. Ferguson or Severini might be useful reference texts. $\endgroup$
    – kurtosis
    Nov 19, 2020 at 1:57
  • 1
    $\begingroup$ @kurtosis, thank you! Regarding normality, I agree. Regarding the statistical argument, what do you mean by the data generation process results from an equilibrium affected by the mean? Could you formulate this mathematically? Could you write down the simplest possible statistical model that would illustrate your point? $\endgroup$
    – Richard Hardy
    Nov 20, 2020 at 16:13

1 Answer 1

Reset to default
2
$\begingroup$

Here is what I believe might be a counterexample if the intuition were a general claim, or at least a result that seems to indicate that the answer to 2. might be "not really". The measure of the precision of an estimator of a certain moment that I use here is the variance.

It is well known that the variance of the sample variance, when sampling from a normal population, is $\frac{2\sigma^4}{n-1}$, and that that of the mean is $\sigma^2/n$.

So, the former is larger if $$\frac{2\sigma^4}{n-1}>\frac{\sigma^2}{n}$$ or $$\sigma^2>\frac{n-1}{2n},$$ which evidently need not be the case.

n <- 10
sigma.sq <- 4/10 # 9/20 or 4.5/10 would be cutoff here

sim.mean.s2 <- function(n){
  x <- rnorm(n, sd=sqrt(sigma.sq))
  xbar <- mean(x)
  s2 <- var(x)
  return(list(xbar, s2))
}

sims <- matrix(unlist(replicate(1e6, sim.mean.s2(n))), nrow=2)

var(sims[1,]) # may also try moments::moment(sims[1,],2, central=T) to simulate population variance, but does not matter at many replications
sigma.sq/n

var(sims[2,])
2*sigma.sq^2/(n-1)
Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Thank you, that is insightful. So essentially there is no good reason to have the general feeling I was having? :) $\endgroup$
    – Richard Hardy
    Nov 12, 2020 at 15:03
  • $\begingroup$ Apparently not - nor to have the one I had, too...:) It might be the case that in the real world $\sigma^2>1$ is common, so that the above inequality often holds so that the intuition does have some empirical support (in any case, the example is somewhat restrictive in that it deals with the normal case and the first two moments only). $\endgroup$
    – Christoph Hanck
    Nov 12, 2020 at 15:12
  • $\begingroup$ Definitely helpful nonetheless! Thanks a lot! $\endgroup$
    – Richard Hardy
    Nov 12, 2020 at 15:15
  • $\begingroup$ Interesting. What if you redo the analysis for standard deviation? Mean and variance are not in the same units so the comparison might be problematic. $\endgroup$
    – fes
    Nov 12, 2020 at 15:41
  • 1
    $\begingroup$ That is a good point! Resorting to asymptotic approximations via the delta method, the standard deviation has asymptotic variance $\sigma^2/2$ (i.e., $\sqrt{n}(S-\sigma)\to_d N(0,\sigma^2/2)$, see hannig.cloudapps.unc.edu/STOR655/handouts/…) while the mean has asymptotic variance $\sigma^2$, which evidently implies a smaller variance of the standard deviation. Should we still call it a "higher" moment if it has the same units as the mean, though? $\endgroup$
    – Christoph Hanck
    Nov 12, 2020 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.