Background
I've found two different bias-corrected estimators for the kurtosis.
The first one is used in various software packages, such as MATLAB, and is called bias-corrected in the respective documentations:
$k_0 = \frac{n-1}{(n-2) (n-3)}\left((n+1) \frac{\frac{1}{n} \sum\limits_{i=1}^n (x_i-\overline{x})^4}{\left(\frac{1}{n} \sum\limits_{i=1}^n (x_i-\overline{x})^2\right)^2}-3 (n-1)\right)+3.$
The second one can be found in Harald Cramér's book Mathematical Methods of Statistics on page 352 and is also the result of using the HStatistic
function from the mathStatica package for Mathematica. It is said to be unbiased:
where $m_i$ is the sample central moment: $m_r = \frac{1}{n} \sum\limits_{i=1}^n (x_i - \overline{x})^r.$
Simulation Results
With both of these estimators being bias-corrected, the average of each of the kurtosis estimates from multiple independent trials should reflect the population kurtosis quite well. I've tried to test this assumption in Mathematica. For the first test, I've used the standard normal distribution and averaged the kurtosis estimates of 100,000 independent trials:
As expected, the average of the kurtosis estimates reflect the population kurtosis quite well. However, when performing the same test with another distribution (in this case the chi-squared distribution with one degree of freedom) the estimates are extremely off:
Question
Can someone provide an explanation why the averages of the bias-corrected kurtosis estimates ($k_0$ and $h_4$) are that off in my simulation? I'm especially surprised by the error of $h_4$, an estimate that is said to be unbiased.