Do third order asymptotics exist? Most asymptotic results in statistics prove that as $n \rightarrow \infty$ an estimator (such as the MLE) converges to a normal distribution based on a second-order taylor expansion of the likelihood function. I believe there's a similar result in Bayesian literature, the "Bayesian Central Limit Theorem", which shows that the posterior converges asymptotically to a normal as $n \rightarrow \infty$
My question is - does the distribution converge to something "before" it becomes normal, based on the third term in the Taylor series? Or is this not possible to do in general?
 A: You are searching for the Edgeworth series aren't you?
http://en.wikipedia.org/wiki/Edgeworth_series#Edgeworth_series
(note that Edgeworth died in 1926, should be in most famous statisticians?  )
A: It is not possible for a sequence to "converge" to one thing and then to another.  The higher-order terms in an asymptotic expansion will go to zero.  What they tell you is how close to zero they are for any given value of $n$.
For the Central Limit Theorem (as an example) the appropriate expansion is that of the logarithm of the characteristic function: the cumulant generating function (cgf).  Standardization of the distributions fixes the zeroth, first, and second terms of the cgf.  The remaining terms, whose coefficients are the cumulants, depend on $n$ in an orderly way.  The standardization that occurs in the CLT (dividing the sum of $n$ random variables by something proportional to $n^{1/2}$--without which convergence will not occur) causes the $m^\text{th}$ cumulant--which after all depends on $m^\text{th}$ moments--to be divided by $(n^{1/2})^m = n^{m/2}$, but at the same time because we are summing $n$ terms, the net result is that the $m^\text{th}$ order term is proportional to $n/n^{m/2} = n^{-(m-2)/2}$.  Thus the third cumulant of the standardized sum is proportional to $1/n^{1/2}$, the fourth cumulant is proportional to $1/n$, and so on.  These are the higher-order terms.  (For details, see this paper of Yuval Filmus for example.)
In general, a high negative power of $n$ is much smaller than a low negative power.
We can always be assured of this by taking a sufficiently large value of $n$.  Thus, for really large $n$ we can neglect all negative powers of $n$: they converge to zero.  Along the way to convergence, departures from the ultimate limit are measured with increasing accuracy by the additional terms: the $1/n^{1/2}$ term is an initial "correction," or departure from the limiting value; the next $1/n$ term is a smaller, more quickly-vanishing correction added to that, and so on.  In brief, the additional terms give you a picture of how quickly the sequence converges to its limit.
These additional terms can help us make corrections for finite (usually small) values of $n$.  They show up all the time in this regard, such as Chen's modification of the t-test, which exploits the third-order ($1/n^{1/2}$) term.
A: Here is an attempt to answer your insightful question. I have seen the inclusion of the 3rd term of the Taylor series to increase the speed of convergence of the series to the true distribution. However, I haven't seen (in my limited experience) the usage of third and higher moments. 
As pointed out by John D. Cook in his blogs (here and here), there hasn't been much work done in this direction, apart from the Berry-Esseen theorem. My guess would be (from the observation in the blog about the approximation error being bounded by $n^{1/2}$), as the asymptotic normality of mle is guaranteed at a rate of converge of $n^{1/2}$ ($n$, being sample size), considering higher moments won't improve on the normality result.
Therefore, I guess, the answer to your question should be no. The asymptotic distribution converges to a normal dist.(by CLT, under regularity conditions of Lindberg's CLT). However, using higher order terms may increase the rate of convergence to the asymptotic distribution. 
A: Definitely not my area, but I'm pretty sure third- and higher-order asymptotics exist. Is this any help?
Robert L. Strawderman. Higher-Order Asymptotic Approximation: Laplace, Saddlepoint, and Related Methods Journal of the American Statistical Association Vol. 95, No. 452 (Dec., 2000), pp. 1358-1364 
