What is an approximate ancillary statistic? In the article Assessing the Accuracy of the Maximum Likelihood Estimator: Observed Versus Expected Fisher Information the authors use the expression "approximate ancillary statistic". This expression is used in a lot of others articles.  Anyone know's the definition of approximate ancillary statistics?
 A: So, better late than never I guess. The textbook "Principles of Statistical Inference" explains this rather cleanly (excerpt here):

A statistic $a$ is approximately ancillary for the estimation of $\theta$ if the asymptotic distribution of $a$ is free of $\theta$

So, it allows you to condition on statistics that are weakly associated with $\theta$ as long as this dependence approaches zero in some suitable sense. 
The observed Fisher information is one such "approximate ancillary" and using it as your variance term for a Wald-type confidence interval about the MLE will approximate a full conditional inference given some true ancillary (should it exist). The beauty of their result is that observed Fisher information is universally available for well-behaved (regular) parametric models and it provides some reassurance that the conditional (post-data) confidence will be approximately equal to the unconditional (pre-data) coverage.
Note however that the authors of the paper you read were not able to theoretically establish the approximate ancillarity of observed information for non-translation models, although they suspect it was the case based on simulations.
