Number of data points needed to estimate the $nth$ moment of a distribution Assuming that I do not know much about an underlying distribution. How many data points does one needs to estimate the $n$th moment?
Is there a formula for the $n$th moment? Is there also formulas for the confidence interval of these parameters? 
If a general formula for $n$th moment is difficult to determine, what do we know about the lower order ones (as far as the number of points needed for estimation) (e.g the first four)?
Any pointer is appreciated.
 A: The raw moments $\mathbb E[X^k]$ all admit unbiased estimators based on an iid sample of size $n$ from the distribution behind the expectation, as for instance$$\frac{1}{n} \sum_{i=1}^n X_i^k$$
As demonstrated in the final page reproduced below Paul Halmos wrote a now famous paper in 1946, called the theory of unbiased estimation, where he gives necessary and sufficient conditions for the existence of unbiased estimators of some expectation based on an iid sample of size $n$ from the distribution behind the expectation.
In particular, he studies the existence of unbiased estimators of the $k$-th centred or central moments$$\mu_k=\mathbb E[(X-\mathbb E\{X\})^k]$$for which he shows

*

*that they only exist when $k\le n$

*that they can only be expressed as a rescaling of the empirical moment$$\hat\mu_k^n = \frac{1}{n}\sum_{i=1}^n (X_i-\bar{X}_n)^k= \frac{1}{n}\sum_{i=1}^n (X_i-\hat\mu_1^n)^k$$when $k\le 3$. For larger values of $k\le n$, the unbiased estimator of $\mu_k$ also depends on $\mu_\ell^n$ for $1\le\ell\le k-1$.

Note however that $\hat\mu_k^n$ is a converging estimator of $\mu_k$ (as $n$ grows to infinity).

Let me also point out that it is always possible to turn biased estimators into unbiased estimators if sequential sampling is available. Using coupling and stopping time and a telescoping sum argument, as demonstrated by Glynn and Rhee (Exact estimation for Markov chain equilibrium expectations.Journal of Applied Probability, 51(A):377–389, 2014.)
A: General Formula
The general formula for the $n\text{th}$ moment for a function $f(x)$  (from https://en.wikipedia.org/wiki/Moment_(mathematics)) is 
$$
\mu_n = \int_{-\infty}^\infty (x - c)^n f(x) dx
$$
where $c = 0$ if you're calculating the first moment (the mean),
and $c = $ the mean otherwise.
With $k$ data points, this can be estimated as
$$
\hat \mu_n = E[ (x - c)^n ] = \frac{\sum_i^k(x_i - c)^n}{k}
$$
For statistical purposes, you'll want to look into the Standardised moments
How much data
At minimum, you need $n$ data points to have an estimate the $n$th moment. You need one to estimate the mean, 2 for the variance, 3 for skew, and so on. Obviously, these will be extremely poor estimates, since these are the absolute minima.
Confidence Intervals
Calculating confidence intervals for higher-order moments in general is tricky. Wright and Herrington (2011) provide a way of estimating them using bootstrap samples.
