# MLE exists and is unique for iid series

Let $$X_1,...,X_n\in R^p$$ be i.i.d. with density, $$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$ where $$c^{-1}(a)=\int_{R^p}\exp(-|\mathbf{x}|^a)d\mathbf{x}$$ and $$|.|$$ is the Euclidean norm.

I want to show that if $$a>1$$, the MLE$$\mathbf{\hat{\theta}}$$ exists and is unique.

Corollary on existence and uniqueness of MLE: Suppose that a rank $$k$$ canonical exponential family distribution has generator $$(\mathbf{T},h)$$ and an open natural parameter space $$\epsilon$$. If the probability distribution of $$\mathbf{T(X)}$$ has a convex support $$C_T$$, then for the data vector $$\mathbf{X}$$ observed as $$\mathbf{x}$$, The MLE exists and is unique iff $$\mathbf{T(x)}\in C_T\backslash \partial C_T$$, the interior of $$C_T$$. In other words:$$P(\mathbf{c}^T\mathbf{T(X)}>\mathbf{c}^T\mathbf{T(x)})>0$$ for any $$\mathbf{c}\neq \mathbf{0}$$.

An alike question: Existence and uniqueness of MLE

Any helps would be appreciated!

• This is NOT an exponential family. Mar 22, 2021 at 7:45

The distribution you are using here is the generalised error distribution (also called the generalised normal distribution) with unit scale and shape parameter $$a>0$$. I take it from your specification that the scale $$a$$ is a fixed value in this analysis, and is therefore not subject to estimation. I will proceed on that basis.

For the data $$\mathbf{x} = (\mathbf{x}_1,...,\mathbf{x}_n)$$ from this distribution the log-likelihood function is:

\begin{align} \ell_\mathbf{x}(\boldsymbol{\theta}) &= \log c(a) - \sum_{i=1}^n ||\mathbf{x}_i-\boldsymbol{\theta}||^a \\[6pt] &= \log c(a) - \sum_{i=1}^n \sum_{r=1}^p |x_{i,r}-\theta_r|^a. \\[6pt] \end{align}

The first and second-order partial derivatives of this function are:

\begin{align} \frac{\partial \ell_\mathbf{x}}{\partial \theta_k} (\boldsymbol{\theta}) &= a \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{x_{i,k}-\theta_k}, \\[12pt] \frac{\partial^2 \ell_\mathbf{x}}{\partial \theta_k \partial \theta_l} (\boldsymbol{\theta}) &= - (a-1) a \cdot \mathbb{I}(k=l) \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}. \end{align}

Now, for all vectors $$\mathbf{z} \in \mathbb{R}^p$$ we have the quadratic form:

$$\mathbf{z}^\text{T} \nabla^2 \ell_\mathbf{x}(\boldsymbol{\theta}) \mathbf{z} = - (a-1) a \times \underbrace{\sum_{i=1}^n \sum_{k=1}^p z_k^2 \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}}_{\text{TERM I}}.$$

For all $$\mathbf{z} \neq 0$$ it can be shown that $$\text{TERM I}>0$$ with probability one. Consequently, if $$a>1$$ is then the log-likelihood function is almost surely negative definite. This means that there is a maximising point for the function, which occurs at the unique solution to the critial point equation:

$$0 = \sum_{i=1}^n \frac{|x_{i,1}-\hat{\theta}_1|^a}{x_{i,1}-\hat{\theta}_1} = \cdots = \sum_{i=1}^n \frac{|x_{i,p}-\hat{\theta}_p|^a}{x_{i,p}-\hat{\theta}_p}.$$

• You wouldn't have the log in the first equation, since it is $exp(−|x−\theta|^a)$. Mar 23, 2021 at 1:51
• @statwoman: Sorry about that --- I misread the initial density function. I have edited the answer to correct the analysis.
– Ben
Mar 23, 2021 at 5:06
• It is quite well-known that if $g(\boldsymbol{\theta})$ is a non-negative convex function of $\boldsymbol{\theta}$ then $g(\boldsymbol{\theta})^a$ is convex for $a \geq 1$. Yet, this does not give a strict convexity, and does not provides us with derivatives fot the optimisation.
– Yves
Mar 23, 2021 at 6:57
• Indeed. But the condition you have specified in your question is $a>1$. I think you can just compute the derivatives from scratch in this case.
– Ben
Mar 23, 2021 at 9:01