Almost sure convergence of the minimum eigenvalue of a sample covariance matrix I was wondering if someone could provide a reference to the following result. Consider the $p\times p $ sample matrix
$$\frac{1}{n} \sum_{i=1}^n x_i x_i',$$
where $x_i$ are i.i.d. $p\times 1$ random vectors (with all the finite moments), we can assume they have mean zero. I conjecture that the minimum eigenvalue of this sample matrix converges almost surely to the minimum eigenvalue of the population matrix $E[x_i x_i']$. 
 A: Let $\Sigma = \mathbb{E}[x_1 x_1^T]$. By the law of large numbers, if we hold the dimension $p$ fixed and let $n \to \infty$,
$$\frac{1}{n} \sum_{i=1}^n x_i x_i^T \stackrel{\mathrm{a.s.}}{\to} \Sigma = \mathbb{E}[x_1 x_1^T]. $$
If we assume the (reasonable) condition that $\Sigma$ has distinct eigenvalues and eigenvectors, then the eigenvalues are a continuous function of $\Sigma$. See The Matrix Cookbook (Petersen and Pedersen), which is linked here) for a reference of this fact. As a result, by the continuous mapping theorem, we have that
$$\lambda_{\min}\left(\frac{1}{n} \sum_{i=1}^n x_i x_i^T\right) \stackrel{\mathrm{a.s.}}{\to} \lambda_{\min}(\Sigma). $$
A final caveat which is related to your question: this result only holds asymptotically when $n$ is many times larger than $p$. More "modern" theory often considers an asymptotic regime where $n, p$ both grow to infinity at a linear rate, meaning $\lim_{n \to \infty} \frac{p}{n} = \gamma \in (0, \infty)$.
In this case, the minimum eigenvalue of the sample covariance matrix does not usually converge to the minimum eigenvalue of the covariance matrix: usually, it is inconsistent (too small asymptotically). See The Smallest Eigenvalue of a Large Dimensional Wishart Matrix by JW Silverstein as a canonical reference, although there has been a lot of additional progress in this field in the past 20 years.
