MLE exists and is unique for iid series

Question

Let $X_1,...,X_n\in R^p$ be i.i.d. with density, $$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$ where $$c^{-1}(a)=\int_{R^p}\exp(-|\mathbf{x}|^a)d\mathbf{x}$$ and $|.|$ is the Euclidean norm.

I want to show that if $a>1$, the MLE$\mathbf{\hat{\theta}}$ exists and is unique.

Corollary on existence and uniqueness of MLE: Suppose that a rank $k$ canonical exponential family distribution has generator $(\mathbf{T},h)$ and an open natural parameter space $\epsilon$. If the probability distribution of $\mathbf{T(X)}$ has a convex support $C_T$, then for the data vector $\mathbf{X}$ observed as $\mathbf{x}$, The MLE exists and is unique iff $\mathbf{T(x)}\in C_T\backslash \partial C_T$, the interior of $C_T$. In other words:$$P(\mathbf{c}^T\mathbf{T(X)}>\mathbf{c}^T\mathbf{T(x)})>0$$ for any $\mathbf{c}\neq \mathbf{0}$.

An alike question: Existence and uniqueness of MLE

Any helps would be appreciated!

Answer 1

The distribution you are using here is the generalised error distribution (also called the generalised normal distribution) with unit scale and shape parameter $a>0$. I take it from your specification that the scale $a$ is a fixed value in this analysis, and is therefore not subject to estimation. I will proceed on that basis.

For the data $\mathbf{x} = (\mathbf{x}_1,...,\mathbf{x}_n)$ from this distribution the log-likelihood function is:

$$\begin{align} \ell_\mathbf{x}(\boldsymbol{\theta}) &= \log c(a) - \sum_{i=1}^n ||\mathbf{x}_i-\boldsymbol{\theta}||^a \\[6pt] &= \log c(a) - \sum_{i=1}^n \sum_{r=1}^p |x_{i,r}-\theta_r|^a. \\[6pt] \end{align}$$

The first and second-order partial derivatives of this function are:

$$\begin{align} \frac{\partial \ell_\mathbf{x}}{\partial \theta_k} (\boldsymbol{\theta}) &= a \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{x_{i,k}-\theta_k}, \\[12pt] \frac{\partial^2 \ell_\mathbf{x}}{\partial \theta_k \partial \theta_l} (\boldsymbol{\theta}) &= - (a-1) a \cdot \mathbb{I}(k=l) \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}. \end{align}$$

Now, for all vectors $\mathbf{z} \in \mathbb{R}^p$ we have the quadratic form:

$$\mathbf{z}^\text{T} \nabla^2 \ell_\mathbf{x}(\boldsymbol{\theta}) \mathbf{z} = - (a-1) a \times \underbrace{\sum_{i=1}^n \sum_{k=1}^p z_k^2 \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}}_{\text{TERM I}}.$$

For all $\mathbf{z} \neq 0$ it can be shown that $\text{TERM I}>0$ with probability one. Consequently, if $a>1$ is then the log-likelihood function is almost surely negative definite. This means that there is a maximising point for the function, which occurs at the unique solution to the critial point equation:

$$0 = \sum_{i=1}^n \frac{|x_{i,1}-\hat{\theta}_1|^a}{x_{i,1}-\hat{\theta}_1} = \cdots = \sum_{i=1}^n \frac{|x_{i,p}-\hat{\theta}_p|^a}{x_{i,p}-\hat{\theta}_p}.$$