3
$\begingroup$

Let $X_1,...,X_n\in R^p$ be i.i.d. with density, $$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$ where $$c^{-1}(a)=\int_{R^p}\exp(-|\mathbf{x}|^a)d\mathbf{x}$$ and $|.|$ is the Euclidean norm.

I want to show that if $a>1$, the MLE$\mathbf{\hat{\theta}}$ exists and is unique.

Corollary on existence and uniqueness of MLE: Suppose that a rank $k$ canonical exponential family distribution has generator $(\mathbf{T},h)$ and an open natural parameter space $\epsilon$. If the probability distribution of $\mathbf{T(X)}$ has a convex support $C_T$, then for the data vector $\mathbf{X}$ observed as $\mathbf{x}$, The MLE exists and is unique iff $\mathbf{T(x)}\in C_T\backslash \partial C_T$, the interior of $C_T$. In other words:$$P(\mathbf{c}^T\mathbf{T(X)}>\mathbf{c}^T\mathbf{T(x)})>0$$ for any $\mathbf{c}\neq \mathbf{0}$.

An alike question: Existence and uniqueness of MLE

Any helps would be appreciated!

$\endgroup$
1
  • $\begingroup$ This is NOT an exponential family. $\endgroup$
    – Xi'an
    Mar 22, 2021 at 7:45

1 Answer 1

1
$\begingroup$

The distribution you are using here is the generalised error distribution (also called the generalised normal distribution) with unit scale and shape parameter $a>0$. I take it from your specification that the scale $a$ is a fixed value in this analysis, and is therefore not subject to estimation. I will proceed on that basis.

For the data $\mathbf{x} = (\mathbf{x}_1,...,\mathbf{x}_n)$ from this distribution the log-likelihood function is:

$$\begin{align} \ell_\mathbf{x}(\boldsymbol{\theta}) &= \log c(a) - \sum_{i=1}^n ||\mathbf{x}_i-\boldsymbol{\theta}||^a \\[6pt] &= \log c(a) - \sum_{i=1}^n \sum_{r=1}^p |x_{i,r}-\theta_r|^a. \\[6pt] \end{align}$$

The first and second-order partial derivatives of this function are:

$$\begin{align} \frac{\partial \ell_\mathbf{x}}{\partial \theta_k} (\boldsymbol{\theta}) &= a \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{x_{i,k}-\theta_k}, \\[12pt] \frac{\partial^2 \ell_\mathbf{x}}{\partial \theta_k \partial \theta_l} (\boldsymbol{\theta}) &= - (a-1) a \cdot \mathbb{I}(k=l) \sum_{i=1}^n \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}. \end{align}$$

Now, for all vectors $\mathbf{z} \in \mathbb{R}^p$ we have the quadratic form:

$$\mathbf{z}^\text{T} \nabla^2 \ell_\mathbf{x}(\boldsymbol{\theta}) \mathbf{z} = - (a-1) a \times \underbrace{\sum_{i=1}^n \sum_{k=1}^p z_k^2 \frac{|x_{i,k}-\theta_k|^a}{(x_{i,k}-\theta_k)^2}}_{\text{TERM I}}.$$

For all $\mathbf{z} \neq 0$ it can be shown that $\text{TERM I}>0$ with probability one. Consequently, if $a>1$ is then the log-likelihood function is almost surely negative definite. This means that there is a maximising point for the function, which occurs at the unique solution to the critial point equation:

$$0 = \sum_{i=1}^n \frac{|x_{i,1}-\hat{\theta}_1|^a}{x_{i,1}-\hat{\theta}_1} = \cdots = \sum_{i=1}^n \frac{|x_{i,p}-\hat{\theta}_p|^a}{x_{i,p}-\hat{\theta}_p}.$$

$\endgroup$
4
  • $\begingroup$ You wouldn't have the log in the first equation, since it is $exp(−|x−\theta|^a)$. $\endgroup$
    – statwoman
    Mar 23, 2021 at 1:51
  • $\begingroup$ @statwoman: Sorry about that --- I misread the initial density function. I have edited the answer to correct the analysis. $\endgroup$
    – Ben
    Mar 23, 2021 at 5:06
  • $\begingroup$ It is quite well-known that if $g(\boldsymbol{\theta})$ is a non-negative convex function of $\boldsymbol{\theta}$ then $g(\boldsymbol{\theta})^a$ is convex for $a \geq 1$. Yet, this does not give a strict convexity, and does not provides us with derivatives fot the optimisation. $\endgroup$
    – Yves
    Mar 23, 2021 at 6:57
  • $\begingroup$ Indeed. But the condition you have specified in your question is $a>1$. I think you can just compute the derivatives from scratch in this case. $\endgroup$
    – Ben
    Mar 23, 2021 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.