Proof for two-sample Hotelling $T^2$ statistic?

Question

I've been reading "A primer of multivariate statistics" by Richard J. Harris, page 546, which shows how to derive the Hotelling $T^2$ statistic, after seeing this related but different question (I have the degrees of freedom $n_1$ and $n_2$ in mine).

The following is left as an exercise for the student as it is outside the scope of a course I did last year:

Let $Y=a^TX$, where $a$ is a constant vector and $X$ is a sample member. Also $\mathcal{N(a^T\mu,a^T\Sigma a)}$

Show that

$\displaystyle\max_{\substack{a}}{\space} t^2(a)=\displaystyle\max_{\substack{a}}\space\frac{n_1n_2[a^T(\bar{x}_1-\bar{x}_2)]^2}{(n_1+n_2)a^TS_Ua}=\frac{n_1n_2}{n_1+n_2}(\bar{x}_1-\bar{x}_2)^TS_U^{-1}(\bar{x}_1-\bar{x}_2)$

which is known as the two-sample Hotelling $T^2$ statistic.

My notes say that the maximum is achieved for:

$a^*=S_U^{-1}(\bar{x}_1-\bar{x}_2)$

I can't find a proof for this despite much searching and it's not something I can work out on my own.

---

My thinking is that we differentiate with respect to $a$, set equal to zero and solve for $a$, and then substitute back into the original?

I can't see how one would differentiate that though and if that is how a proof is done?

Answer 1

Unless I am missing something, this can be seen from using Cauchy-Schwarz & Spectral Decomposition (as shown in pages 78-80 in Applied Multivariate Statistical Analysis, by Richard A. Johnson and Dean W. Wichern, 6th edition).

First Cauchy-Schwarz: For two $p \times 1$ vectors $\mathbf{b}$ and $\mathbf{d}$, $$ (\mathbf{b}' \mathbf{d})^2 \leq (\mathbf{b}' \mathbf{b})(\mathbf{d}' \mathbf{d}),$$ with equality iff $\mathbf{b} = c \, \mathbf{d}$ for some constant $c$.

If $\mathbf{B}$ is a symmetric positive definite matrix, using spectral-decomposition, define $\mathbf{B}^{1/2}$ and $\mathbf{B}^{-1/2}$ appropriately. Then, applying the C-S inequality above to the vectors $\mathbf{B}^{1/2}\mathbf{b}$ and $\mathbf{B}^{-1/2}\mathbf{d}$, one gets a generalized version: $$ (\mathbf{b}' \mathbf{d})^2 \leq (\mathbf{b}' \mathbf{B} \mathbf{b})(\mathbf{d}' \mathbf{B}^{-1}\mathbf{d}),$$ with equality iff $\mathbf{b} = c \, \mathbf{B}^{-1}\mathbf{d}$ for some constant $c$.

Now if $\mathbf{b} \neq \mathbf{0}$, and since $\mathbf{B}$ is symmetric positive definite, we can divide both sides by $\mathbf{b}' \mathbf{B} \mathbf{b}$ to get: $$ \frac{(\mathbf{b}' \mathbf{d})^2 } {\mathbf{b}' \mathbf{B} \mathbf{b}} \leq \mathbf{d}' \mathbf{B}^{-1}\mathbf{d}$$.

Appropriate replacements by corresponding vectors and matrices should give the result stated.

HTH.

Answer 2

yes, you simply differentiate with respect to $a$, set to zero, and back-substitute. First note that the objective, $t^2(a)$, is positively scale-invariant with respect to $a$. That is, for $k>0$, $t^2(ka) = t^2(a)$. This allows you to be somewhat sloppy when computing the derivative, since any (positive) constant can be dropped.

The derivative simply follows the quotient rule of calculus. The derivative (with respect to $a$) of the numerator of $t^2$ is, up to scale, $a^{\top}\left(\bar{x}_1 - \bar{x}_2\right)$. The derivative of the denominator is, up to scale, $S_U a$. When using the 'lo-dee-hi minus hi-dee-lo etc' rule, the derivative is something like $c_1 a^{\top}\left(\bar{x}_1 - \bar{x}_2\right) - c_2 S_U a$ for positive constants $c_i$.