2
$\begingroup$

I know the proof but I'm unclear on one thing.

Cauchy-Schwarz inequality: Given X,Y are random variables, the following holds:

$$ (E[XY])^2 \le E[X^2]E[Y^2] $$

Proof

Let $$ u(t) = E[(tX - Y)^2] $$

Then:

$$ t^2E[X^2] - 2tE[XY] + E[Y^2] \ge 0 $$

This is a quadratic in $t$. Thus the discriminant must be non-positive. Therefore:

$$ (E[XY])^2 - E[X^2]E[Y^2] \le 0 $$

Why must the discriminant be non-positive?

$\endgroup$
1
  • 2
    $\begingroup$ Hint: apply the quadratic formula. $\endgroup$
    – whuber
    Commented Dec 14, 2018 at 23:24

3 Answers 3

2
$\begingroup$

Let $p(t):=at^2+bt+c$ be a second degree polynomial. Then the roots of $p(t)$ are: $t_{1,2}:=\frac{-b\pm \sqrt{D}}{2a}$ where $D:=b^2-4ac$. This defines the discriminant $D$ of a polynomial. if $D\geq 0$, this implies that both roots (possibly repeated) are real. In particular if $D>0$, there are two distinct real roots, and you can easily prove that this implies $p(x)<0$ for at least one $x$ (consider the derivative at either of the roots).

$D\leq 0$ this implies both roots are imaginary, so that either $p(t)\leq 0$ or $p(t)\geq 0$ for all $t$. Similarly if $p(t)\geq 0$ or $p(t)\leq 0$ for all $t$, this implies $p(t)$ can have at most one distinct real root.

If you now define $p(t):=t^2E[X^2]-2tE[XY]+E[Y^2]$, you've proven thus far that $p(t)\geq 0$, since $u(t)\geq 0$. It follows that the discriminant $D$ of $p$ must be non-positive, e.g. $D(t)\leq 0$.

$\endgroup$
2
$\begingroup$

The quadratic in question must have complex roots, or at best two identical real roots; else there would be an interval $I$ such that for all $t\in I$, the random variable $tX-Y$ has negative mean-square value $E[(tX-Y)^2]$ which is impossible. If the discriminant is negative, then the roots of the quadratic are complex-valued (remember $\frac{-b \pm \sqrt{b^2-4ac}}{2a}$ where $b^2-4ac$ is the discriminant?) and so everything works as expected.

What if the quadratic has two identical real-valued roots and so has value $0$ for one specific $t$? Well, in that case, $tX-Y$ has mean-square value $0$, that is, $tX$ is a perfect predictor of $Y$ and it must be that the correlation coefficient $\rho_{X,Y}$ has value $\pm 1$.

$\endgroup$
1
$\begingroup$

I explained this to myself by visualising the graphs of a repeated root quadratic equation. For any $x \in R$, $(ax + b)^2 \ge 0$ and graph of $y = (ax + b)^2$ must be a parabola above the x-line or touching the x line. This means that the equation $y = (ax + b)^2$ either has no roots (graph not touching/intersecting x-line at all) or has at most a repeated root (graph touching x-line). Therefore, the discriminant of $y = (ax + b)^2$ must satisfy $D \le 0$.

Going back to the stats question, $(tX + Y)^2$ must be a positive number for any $t \in R$. Therefore, $y = (tX + y)^2$ must be a parabola above the x-line or at most touching the x line. Similarly to above, this means $D \le 0$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.