How to show $\rho(\beta_0, \beta_1) \leq 0$? Consider a simple linear regression $Y_i = \beta_0 + \beta_1x_i + \epsilon_i$ where each $\epsilon_i \sim N(0, \sigma^2)$.
The solution to the linear regression problem is given by $\pmb{\beta} = (\pmb{X}^{T}\pmb{X})^{-1}\pmb{X}^{T}\pmb{Y}$ and the variance-covariance matrix is $\text{Var}(\pmb{\beta}) = \sigma^2 (\pmb{X}^{T}\pmb{X})^{-1}$.
Provided $x_i \geq 0$ for each index $i$, I want to show that $\rho(\hat{\beta_0}, \hat{\beta_1}) \leq 0,$ where $\rho(\cdot, \cdot)$ denotes the correlation.

I tried expanded the correlation definition, and I think enough to show covariance is negative. However, I'm having trouble doing so. I've used the definition of correlation, and I think I need to make use of the fact that $x_i \geq 0$, but I'm not able to get anywhere.
Can anyone please help me?
 A: I will set forth the argument doing as little algebra as possible, working from basic definitions.
$\mathbf X$ is the model matrix: its rows are numerical representations of the observations and its columns are the regressor variables recorded for each observation. (The vector $y$ separately records the values of the response variable.) Usually $X$ also includes a constant column used to model the constant, or "intercept," term.
In Ordinary Least Squares regression there is just one regressor variable, often denoted $\mathbf x = (x_1,x_2,\ldots, x_n).$  Thus, placing the constant column first in the model matrix,
$$\mathbf{X} = \pmatrix{1&x_1 \\ 1&x_2 \\ \vdots & \vdots \\ 1 & x_n}\text{ and }\mathbf y = \pmatrix{y_1 \\ y_2 \\ \vdots \\ y_n}.$$
We can proceed only by supposing the $x_i$ aren't all the same number (which is implicit in the question).  According to your formula $\sigma^2 \left(\mathbf X^\prime \mathbf X\right)^{-1},$ the variance of the coefficient estimates $\hat\beta=(\hat\beta_0,\hat\beta_1)$ is a positive multiple ($\sigma^2$) of the matrix inverse of
$$\mathbf  X^\prime \mathbf X = \pmatrix{1 & 1 & \ldots & 1 \\ x_1 & x_2 & \ldots & x_n}\pmatrix{1&x_1 \\ 1&x_2 \\ \vdots & \vdots \\ 1 & x_n} = \pmatrix{n & \sum_{i=1}^n x_i \\ \sum_{i=1}^n x_i & \sum_{i=1}^n x_i^2},$$
as you can compute using the rule of matrix multiplication.  (If you track through the remainder of this analysis closely, you can confirm that only one of these calculations is actually needed: the upper right corner of this matrix product is $(1,1,\ldots,1)(x_1,x_2,\ldots,x_n)^\prime = \sum x_i.$)
The inverse of $\mathbf  X^\prime \mathbf X$ is its adjugate divided by its determinant, where
$$\operatorname{ad} \pmatrix{a&b\\c&d} = \pmatrix{d&-b\\-c&a}$$
and
$$\det \pmatrix{a&b\\c&d} = ad-bc.$$
You can readily confirm this by multiplication:
$$\left[\frac{1}{\det \pmatrix{a&b\\c&d}} \operatorname{ad} \pmatrix{a&b\\c&d}\right]\, \pmatrix{a&b\\c&d} = \frac{1}{ad-bc}\pmatrix{ad-bc & 0 \\ 0 & ad-bc}=\pmatrix{1&0\\0&1}.$$
We need to be concerned about the sign of the determinant.  Since for any 2-vector $\mathbf v$ it is the case that
$$\mathbf v^\prime \left(\mathbf X^\prime \mathbf X\right)\mathbf v = \mathbf w^\prime \mathbf w = ||\mathbf w ||^2 \ge 0,$$
where $\mathbf w = \mathbf {X v},$ it follows from this (and from the non-constancy of $\mathbf x$) that $\mathbf X^\prime \mathbf X$ is positive definite, whence its determinant cannot be negative.  This is a nice argument because it requires no calculation at all.
(Alternatively, if you are a glutton for doing algebra, you may compute
$$\det \mathbf X^\prime \mathbf X = n\sum_{i=1}^n x_i^2 - \left(\sum_{i=1}^n x_i\right)^2 = n^2 \operatorname{Var}(\mathbf x) \gt 0$$
which shows the same thing.)
The off-diagonal terms of the inverse of $\mathbf X^\prime \mathbf X$ therefore are positive multiples of $-\sum_{i=1}^n x_i.$  When you assume all the $x_i$ are non-negative (and at least one is positive), this expression clearly is negative.  Consequently, according to the formula you cite, the covariance (and hence the correlation) of $\hat\beta_0$ and $\hat\beta_1$ must be negative, QED.
In reviewing the argument, please notice that only one extremely easy calculation was involved: we had to sum the $x_i$ when computing $\mathbf X^\prime \mathbf X.$
Geometrically, when you wiggle a line through the scatterplot of points $(x_i,y_i)$ in an effort to approximate those points, tilting the line up increases the slope $\hat\beta_1$ with a consequent decrease in the intercept $\hat\beta_0,$ which (since the $x_i$ are nonnegative) lies to the left of all the points in the scatterplot.  That's what it means to be negatively correlated.

A: Let's say that $x=(1,2,3)$. Then the $X$ matrix is
$$\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}$$
and $X^TX$ is
\begin{align*}
X^TX&=\begin{bmatrix}1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix}
\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}=
\begin{bmatrix} 3 & 6 \\ 6 & 14 \end{bmatrix} \\
&= \begin{bmatrix} n & \sum_ix_i \\ \sum_ix_i & \sum_ix_i^2 \end{bmatrix}
\end{align*}
Finally, $(X^TX)^{-1}$ is
\begin{align*}
(X^TX)^{-1}&=\frac16\begin{bmatrix} 14 & -6 \\ -6 & 3 \end{bmatrix} \\
&= \frac{1}{n\sum_ix_i^2-(\sum_ix_i)^2}\begin{bmatrix}\sum_ix_i^2 & -\sum_ix_i \\ -\sum_ix_i & n \end{bmatrix}
\end{align*}
In general (see Seber & Lee, Linear Regression Analysis, John Wiley & Sons, 2003, Example 4.6), putting $X=[1_n,X_1]$:
$$X^TX=\begin{bmatrix}n & n\bar{x}^T \\ n\bar{x} & X_1^TX_1 \end{bmatrix},
\quad
(X^TX)^{-1}=\begin{bmatrix} \frac1n+\bar{x}^TV^{-1}\bar{x} & -\bar{x}^TV^{-1} \\ -V^{-1}\bar{x} & V^{-1} \end{bmatrix} $$
where $V=\tilde{X}^T\tilde{X}$ and $\tilde{X}$ has tipical element $\tilde{x}_{ij}=x_{ij}-\bar{x}_j$.
Is this enough?
