Is a "weighted" covariance/variance calculation still positive semi definite?

Question

Suppose I want to calculate a sample variance/covariance matrix:

cov(x) = $ \frac{\sum _{i=1} ^k (x_i - \overline{x})(x_i - \overline{x})^T}{k} $

I was wondering, if I calculate

$ \frac{\sum _i n_i(x_i - \overline{x})(x_i - \overline{x})^T}{k}$ , where n$_i>0$ are scalar

Would the resulting matrix still be positive semi definite?

Thank you for any insight on this.

Answer 1

Yes. Each $S_i = (x_i-\bar x)(x_i - \bar x)^T$ is a p.s.d. matrix, so any combination with non-negative weights will be p.s.d., as well.

To wit, consider, for an arbitrary compatible (same dimension) $a\in \mathbb{R}^p, p = {\rm dim}\, x$, the quadratic form $a^T S_i a$:

$$ a^T S_i a = a^T (x_i-\bar x)(x_i - \bar x)^T a = [a^T (x_i-\bar x)] \, [(x_i - \bar x)^T a] = [a^T (x_i-\bar x)]^2 \ge 0 $$

since the term in the square brackets is simply a scalar. The worst thing that can happen is that the vector $a$ is orthogonal to $x_i-\bar x$.