The test statistic in the likelihood ratio test for nested linear models

Question

Imagine that we have a family of probability disributions with p.d.f $f_{\theta}(z)$ where $\theta \in \Theta$. We also know that there is a linear dependence between parameters. As a consequence we can restrict to a nested model with p.d.f $f_{\theta}(z)$, where $\theta \in \Theta_{0} \subseteq \Theta$.

Formally we have such a situation: \begin{align} \Theta \subseteq \mathbb{R}^p,~~~~ h:\mathbb{R}^p \to \mathbb{R}^{p-q},~~~~ \Theta_{0} = \{\theta \in \Theta : h(\theta)=0\}. \end{align} where h is a linear map onto $\mathbb{R}^{p-q}$ so we can say that: \begin{align} h(\theta) = A\theta = 0, \end{align} where $A$ is a $(p-q) \times p$ matrix of a linear map $h$.

As a result we can say that $\Theta_{0} \subseteq \mathbb{R}^q$. HERE BEGINS MY PROBLEM. I would be very grateful if someone could tell me why we can conclude now that \begin{align} \sup \limits_{\theta \in \Theta_{1}}f_{\theta}(z) \overset{\huge{?}}{=} \sup \limits_{\theta \in \Theta}f_{\theta}(z). \end{align} Consequently the test statistic of a likelihood ratio test is \begin{align} \lambda(z) = \frac{\sup_{\theta \in \Theta_{1}}f_{\theta}(z)}{\sup_{\theta \in \Theta_{0}}f_{\theta}(z)}\overset{\huge{?}}{=} \frac{\sup_{\theta \in \Theta}f_{\theta}(z)}{\sup_{\theta \in \Theta_{0}}f_{\theta}(z)}. \end{align}

Answer 1

There is some confusing (to me) notation in your question. If the parameter vector $\theta$ belongs to the $p$-dimensional space $\Theta$ then it is not $\theta$ again that belongs to the $q$-dimensional subspace $\Theta_0$. This is not an issue of values-taken, but of dimensions. (Also it seems that you suddenly change the parameter vector subscript from $0$ to $1$ in the $\sup$ expression).

I understand your question as follows: By assumption, part of the parameters is a function of the rest. Denote the sub-vector containing the "dependent" elements of the $\theta$ vector as $\theta_{p-q} \subseteq \mathbb{R}^{p-q}$, and the "independent" parameter vector as $\theta_q\subseteq \mathbb{R}^{q}$. We then have $\theta_{p-q} = g(\theta_q)$. Then your likelihood function can be written as

$$f(z;\theta) = f(z;\theta_{p-q} , \theta_q) = f(z;g(\theta_q) , \theta_q) $$

Given the assumption of functional dependence among parameters, I believe you don't need to "prove" in any other way that the RHS is the same as the LHS, and so that they will have the same extrema.

The $\sup$-equality expression that you questionmark may be misleading because it could be taken to imply that it holds, irrespective of the existence of the functional dependence, because it does not "reveal" the functional dependence among the parameters. A more clear expression would be

$$ \sup \limits_{\{\theta_{p-q}\}\cup \{\theta_q\} =\theta \in \Theta}f(z;\theta_{p-q} , \theta_q) \;\;\text{s.t.}\;\; \theta_{p-q} = g(\theta_q) $$ $$=\sup \limits_{\theta_q \in \Theta_{0}}f(z;g(\theta_q) , \theta_q) $$

In other words, you just transform a maximization problem with equality constraints into an unconstrained maximization problem by inserting the constraint into the objective function.