# Variation of Skewness and Excess Kurtosis

I apologize ahead of time if this a trivial question. I am building probability models for games - and in order to test the game code, I am comparing the results of large iterations of games with the original probability models. Specifically, I am calculating Standard Deviation, Skewness and Excess Kurtosis and using these calculations for verification. The sticking point lies in calculating appropriate confidence intervals since the underlying distributions are almost always not normal. I have been searching for quite some time for a solution - and the only thing which seems apparent is the BCa bootstrapping method, though I am not sure that this is really appropriate or necessary. It seems to me that there ought to be a way, given the original distribution, to properly calculate the expected variation not only of the mean, but also of the standard deviation, skewness and kurtosis. Is this possible?

• Due to some helpful comments, I am going to try to rephrase this (you may ignore the comments that I tried to make in response as they are a bit disjointed):

Okay, let me see if I can rephrase this thought taking away the focus of the games since that is only relevant in terms of context. If I have a discrete distribution which is not normal - and then I proceed to recreate this distribution through some sort of Monte Carlo process - what is the best way to check that the Monte Carlo process is accurately recreating the original distribution? My thought was that since a given distribution is characterized by its moments, I could use this characterization. The problem with this approach is how to estimate whether the observed error in the moments (from the Monte Carlo simulation) is significant. It seems to me that there should be some way to do this.

• Hi there, do you mean that you are looking at the distributions of final scores for games? I'm not following what you mean by results of large iterations of games. – Michelle Feb 16 '12 at 9:23
• You don't often see skewness and kurtosis used for this purpose because their standard errors are proportional to the sixth and eighth central moments of the distribution, respectively, and those can get huge. A question that describes what you want to know and the information you have (or can readily obtain), rather than focusing on a single technique you have adopted, would broaden the discussion and perhaps get you some more helpful replies. – whuber Feb 16 '12 at 15:38

If you have a discrete distribution, you would want to use Pearson $\chi^2$ test: $$X^2 = \sum_{k=1}^K \frac{(n_k-np_k)^2}{p_k}$$ where $p_k$ is the known population probability, and $n_k$ is the sample frequency. Frankly, I doubt you will get much value of this test, though, as with a single testing, you will never know if you have genuine differences, or just type I error. What you could do though is to break your series into shorter chunks, compute either the $X^2$ or $p$-values, and test them against the target distribution ($\chi^2_{K-1}$ for the test statistic, $U[0,1]$ for the $p$-values) using Kolmogorov-Smirnov test.
I totally agree with whuber that working with moments is pointless. Moreover, for high skewness and kurtosis, the sample estimates may not be informative: there are upper limits on these sample statistics, so a kurtosis of say 10000 will never materialize in a sample of $n=50$ observations. See Cox (2010). This is a mathematical curiosity that I have never seen in my graduate coursework, although I now think it is an important caution against using the sample moments indiscriminately.
• As @whuber explained, variability in skewness depends on the 6th order moments. With some moderate paper-and-pencil work, you should be able to derive the variance of the sample third moment (unstandardized, without dividing by the standard deviation) to construct an asymptotic $z$-test. Derivations of this kind have been done to test normality (en.wikipedia.org/wiki/Jarque-Bera_test). – StasK Feb 17 '12 at 16:19