Mainly for pedagogical reasons, I'm considering the "simple" one dimensional model: $$x=\theta+\epsilon$$ where $\epsilon$ has a known distribution $p$ that is independent of $\theta$. This distribution can be anything, just assuming some regularity conditions, compact support... something that leaves a lot of freedom for the choice of $p$. Somehow I'm looking for properties that hold for "almost any $p$", ignoring special cases with strong underlying algebraic properties (like in an exponential family).
Consider an independent sample $X=(x_1,x_2,x_3,...,x_n)$.
I'm looking for a general way to construct the best unbiased estimator if it exist. Not asymptotically I mean but for fixed finite $n$. It happens to be a bit more than difficult I thought. The questions I face are:
- what is a minimal sufficient statistics (is it just reordered $X$)?
- the MLE has a constant bias (see note), is debiased MLE the best unbiased estimator?
- how to use an ancillary statistics to find the best unbiased estimator? Would it yield the same estimator?
I guess I'm not the first person on earth to ask myself these questions. Have you done this kind of analysis, do you know results, the "right" method, or do you know of an article that studied something like it?
Note: proof that MLE $\hat\theta$ has constant bias:
$E_\theta(\hat\theta)=E_\theta(\arg\max_t \prod_ip(x_i-t))=E_\theta(\arg\max_t \prod_ip(\epsilon_i-(t-\theta)))\\=E_\theta(\arg\max_u \prod_ip(\epsilon_i-u)+\theta)=E_0(\hat\theta)+\theta=c+\theta $
When $p$ is symetric around 0, this bias is 0: since $-\epsilon$ has the same distribution as $\epsilon$: $E_0 (\hat\theta)=E_\theta(\arg\max_t \prod_ip(-\epsilon_i-t))\\=E_\theta(\arg\max_t \prod_ip(\epsilon_i+t))=E_\theta(-\arg\max_u \prod_ip(\epsilon_i-u))=-E_0(\hat\theta)$