What are some illustrative applications of empirical likelihood?

Question

I have heard of Owen's empirical likelihood, but until recently paid it no heed until I came across it in a paper of interest (Mengersen et al. 2012).

In my efforts to understand it, I have gleaned that the likelihood of the observed data is represented as $$L = \prod_i p_i = \prod_i P(X_i=x) = \prod_i P(X_i \le x) - P(X_i \lt x)$$ , where $\sum_i p_i = 1$ and $p_i > 0$.

However, I have been unable to make the mental leap connecting this representation with how it can be used to make inferences about observations. Perhaps I am too rooted in thinking of a likelihood w.r.t parameters of a model?

Regardless, I have been searching Google Scholar for some paper employing empirical likelihood that would help me internalize the concept... to no avail. Obviously, there is Art Owen's book on Empirical Likelihood, but Google Books leaves out all the yummy bits and I'm still in the slow process of getting an inter-library loan.

In the meantime, can somebody kindly point me to papers and documents that clearly illustrate the premise of empirical likelihood and how it is employed? An illustrative description of EL itself would also be welcome!

Answer 1

I can think of no better place than Owen's book to learn about empirical likelihood.

One practical way to think about $L = L(p_1, \ldots, p_n)$ is as the likelihood for a multinomial distribution on the observed data points $x_1, \ldots, x_n$. The likelihood is thus a function of the probability vector $(p_1, \ldots, p_n)$, the parameter space is really the $n$-dimensional simplex of probability vectors, and the MLE is putting weight $1/n$ on each of the observations (supposing they are all different). The dimension of the parameter space increases with the number of observations.

A central point is that empirical likelihood gives a method for computing confidence intervals by profiling without specifying a parametric model. If the parameter of interest is the mean, $\mu$, then for any probability vector $p = (p_1, \ldots, p_n)$ we have that the mean is
$$\mu(p) = \sum_{i=1}^n x_i p_i,$$ and we can compute the profile likelihood as $$L_{\text{prof}}(\mu) = \max \{ L(p) \mid \mu(p) = \mu \}.$$ Then we can compute confidence intervals of the form $$I_r = \{ \mu \mid L_{\text{prof}}(\mu) \geq r L_{\text{prof}}(\bar{x}) \}$$ with $r \in (0,1)$. Here $\bar{x}$ is the empirical mean and $L_{\text{prof}}(\bar{x}) = n^{-n}$. The intervals $I_r$ should perhaps just be called (profile) likelihood intervals since no statement about coverage is made upfront. With decreasing $r$ the intervals $I_r$ (yes, they are intervals) form a nested, increasing family of confidence intervals. Asymptotic theory or the bootstrap can be used to calibrate $r$ to achieve 95% coverage, say.

Owen's book covers this in detail and provides extensions to more complicated statistical problems and other parameters of interest.

Answer 2

In econometrics, many applied papers start with the assumption that $$ E[g(X,\theta)] = 0 $$ where $X$ is a vector of data, $g$ is a known system of $q$ equations, and $\theta \in \Theta \subseteq \mathbb{R}^p$ is an unknown parameter, $q \geq p$. The function $g$ comes from an economic model. The goal is to estimate $\theta$.

The traditional approach, in econometrics, for estimation and inference on $\theta$ is to use generalized method of moments: $$ \hat{\theta}_\text{GMM} = \text{argmin}_{\theta \in \Theta} \; \bar{g}_n(\theta) 'W \bar{g}_n(\theta) $$ where $W$ is a positive definite weighting matrix and $$ \bar{g}_n(\theta) := \frac{1}{n} \sum_{i=1}^n g(X_i,\theta). $$ Empirical likelihood providers an alternative estimator to GMM. The idea is to enforce the moment condition as a constraint when maximizing the nonparametric likelihood. First, fix a $\theta$. Then solve $$ L(\theta) = \max_{p_1,\ldots,p_n} \; \prod_{i=1}^n p_i $$ subject to $$ \sum_{i=1}^n p_i=1, \qquad p_i \geq 0, \qquad \sum_{i=1}^n p_i \cdot g(X_i,\theta) = 0. $$ This is the `inner loop'. Then maximize over $\theta$: $$ \hat{\theta}_\text{EL} = \text{argmax}_{\theta \in \Theta} \; \log L(\theta). $$ This approach has been shown to have better higher order properties than GMM (see Newey and Smith 2004, Econometrica), which is one reason why it is preferable over GMM. For additional reference, see the notes and lecture by Imbens and Wooldridge here (lecture 15).

There are of course many other reasons why EL has garnered attention in econometrics, but I hope this is a useful starting place. Moment equality models are very common in empirical economics.

Answer 3

In survival analysis, the Kaplan-Meier curve is the most famous non-parametric estimator of the survival function $S(t) = Pr(T > t)$, where $T$ denotes the time-to-event random variable. Basically, $\hat{S}$ is a generalisation of the empirical distribution function which allows censoring. It can be derived heuristically, as given in most practical textbooks. But it can also be formally derived as a maximum (empirical) likelihood estimator. Here are more details.