# Interpretation of the Maximum Likelihood Estimator formula

Let $\mathbf{X} = (X_1, \ldots, X_n)$ be an i.i.d sample from the parametric family of distributions $\mathcal{P} = \{P_\theta: \theta \in \Theta \subset \mathbb{R}^k\}$ ($X_i \sim P_\theta$ are random variables, not just real numbers). In my statistics textbook there is the following formula for the Maximum Likelihood Estimator $\hat\theta(\mathbf{X})$ (which is also a random variable, not an estimate):

$$\hat\theta(\mathbf{X}) = \underset{\theta \in \Theta}{\operatorname{argmax}} L(\mathbf{X}; \theta),$$ where $L(\mathbf{X}; \theta)$ is a random variable which is obtained from an ordinary (numerical) likelihood function $L(\mathbf{x},θ)$ by substituting random vector $\mathbf{X}$ on place of numerical vector $\mathbf{x}$.

I am concerned about the fact that random vector $\mathbf{X} = (X_1, \ldots, X_n)$ implicitly depends on $\theta$ because $X_i \sim P_\theta$. This makes it difficult to interpret the maximization process.

Should I treat random variables $(X_1, \ldots, X_n)$ as constants in the maximization process?

If not, what will be the correct interpretation of the above formula?

• The concept of MLE is: assuming the observed $\pmb{x}$ is "most-likely" to occur, we wish to find the value of $\theta$ which maximizes the likelihood $L(\pmb{x};\theta)$. Thus the MLE of $\theta$ will be a function of $\pmb{x}$. Now for further analysis, if we wish to find distribution or confidence interval of the MLE of $\theta$, we replace $\pmb{x}$ by $\pmb{X}$. May 19, 2018 at 21:18

To make the distinction between $\theta$ which is the running argument in the likelihood function $L({\mathbf X};\theta)$ and $\theta_0$ which is the parameter at the source of the generation of $\mathbf X$ [assuming the model is true and there is such a $\theta_0$], one can write $\mathbf X$ as the deterministic transform$${\mathbf X}={\mathbf \Psi}(\epsilon,\theta_0)\sim P_{\theta_0}$$where $\epsilon$ is a "white noise" in the sense it is a random vector with a fixed distribution, of dimension possibly larger than the dimension of $\mathbf X$ (to keep the representation as general as possible). The likelihood function then writes as$$L({\mathbf \Psi}(\epsilon,\theta_0);\theta)$$which can be optimised in $\theta$ [and not in $\theta_0$ since, while ${\mathbf X}={\mathbf \Psi}(\epsilon,\theta_0)$ is observed, $\epsilon$ is not.
• Thanks! You mean that I should write $X_i \sim P_{\theta_0}$ (where $\theta_0$ is true parameter) instead of $X_i \sim P_\theta$ in my post above to use the formula for $\hat\theta(\mathbf{X})$? It seems that in this case $\mathbf{X}$ will not depend on varying parameter $\theta$ and the formula will be correct. May 20, 2018 at 13:20
• (+1) Exactly so! This "tale of two $\theta$'s" is also at the core of the EM algorithm. May 20, 2018 at 13:57
Although Xi'an's answer is correct I want to note that in most cases we use model where random vector $\mathbf{X}=(X_1, \ldots, X_n)$ doesn't depend on any parameter at all, only its distribution depends on $\theta$. In such cases the maximization process is easily interpreted. You can find details here.