3
$\begingroup$

I would like to find maximum of the following function:

$$I = \max_{a\in \mathbb{R}^p} \frac{(a'\hat{\beta})^2}{S^2a'(X'X)^{-1}a},$$

where $X$ is a design matrix and of course $Y$ is normally distributed $N(X\beta, \sigma^2I).$

Trying to do it the standard way, i.e. differentiating the quotient doesn't work well, I can't solve the resulting equation for $a$. Is it manageable to do it through differentiation?

I tried to do it using Cauchy-Schwarz inequality. I obtained:

$$I = \frac{1}{S^2} \max_{a\in \mathbb{R}^p} \frac{a'(X'X)^{-1}X'YY'X(X'X)^{-1}a}{a'(X'X)^{-1}X'X(X'X)^{-1}a}.$$

And now by Cauchy-Schwarz inequality what I get is

$$\frac{1}{S^2} \frac{(c'Y)^2}{c'c} \leq \frac{1}{S^2} \frac{c'cY'Y}{c'c} = \frac{Y'Y}{S^2},$$

where $c = X(X'X)^{-1}a$

But the equality in Cauchy-Schwarz inequality holds, when there exists $d\in \mathbb{R}$ such that $c = dY$ but it requires invertibility of $(XX').$

I would appreciate any input.

$\endgroup$
1
  • $\begingroup$ $(XX’)$ can’t be invertible because it is not full rank. $\endgroup$
    – treskov
    Nov 17, 2019 at 23:01

2 Answers 2

2
$\begingroup$

Using that $\hat\beta= (X^TX)^{-1} X^T Y$ so $(a^T \hat\beta )^2=a^T (X^TX)^{-1} X^T Y Y^T X (X^TX)^{-1} a$, you can write $I$ as $$ I= max_{a\in\mathbb{R}^p}\frac{a^T C a}{a^T B a} $$ where $C=(X^TX)^{-1} X^T Y Y^T X (X^TX)^{-1}$ and $B=(X^TX)^{-1}$, where we left out $S^2$ to simplify, since it is just a constant so does not influence the solution. Then you can apply the answer given to your other question.

$\endgroup$
1
$\begingroup$

Consider the following result, discussed at https://math.stackexchange.com/q/1226524/321264 :

If $B$ is a symmetric positive definite matrix and $x,d$ are real vectors, then $$\max_{x\ne 0}\frac{(x^Td)^2}{x^TBx}=d^T B^{-1}d \tag{1}$$

The proof is based on the following generalization of Cauchy-Schwarz inequality:

$$(x^T d)^2\le (x^TBx)(d^TB^{-1}d) \tag{2}$$

And to prove $(2)$, one can apply the usual Cauchy-Schwarz inequality to $B^{1/2}x$ and $B^{-1/2}d$.

I assume $X$ is of full column rank, whence $X^TX$ is a positive definite matrix.

Using $(1)$, your $I$ reduces to

$$I=\frac{\hat\beta^T(X^TX)\hat\beta}{S^2}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.