8
$\begingroup$

I understand that a covariance matrix with all diagonal elements equal, and all off-diagonal elements also equal (but different to the diagonal elements) is called "spherical". I am curious about the origin of this term. Presumably it has a geometric interpretation. Is it as simple as meaning that for a sphere, the distances from the centre to any point on the surface is equal ? Does it also mean that the distribution of points on the surface is uniform ?

$\endgroup$
1
  • 4
    $\begingroup$ The matrix $\Sigma$ represents a quadratic form defined by the map $x\to x^\prime \Sigma x = Q(x)$ ($x$ is a vector of course). Plot a level surface of this form. By definition, it is the locus of points $x$ satisfying $Q(x)=\lambda$ for some fixed (positive) value $\lambda.$ If that's troublesome to do, carry it out for 2D vectors $X.$ $\endgroup$
    – whuber
    Commented Dec 29, 2023 at 18:53

2 Answers 2

6
$\begingroup$

This form of covariance matrix is actually more "elliptical" than "spherical"

I have not heard of this form of matrix being called a "spherical" covariance matrix, and indeed, that is not an entirely accurate characterisation in my view. To answer your question I will first give some background about this matrix form in statistics and then I will show how the matrix form leads to a property relating to an ellipsoid (with a spherical shape as a special case).

The type of matrix you are referring to is examined in O'Neill (2020) where it is called a "double-constant matrix". Details of the properties of this form of matrix, and its uses in statistics, can be found in that paper. In general, the double-constant matrix has the following mathematical form:

$$\mathbf{\Sigma} \equiv \mathbf{\Sigma}(a,t) \equiv (a-t) \cdot \mathbf{I}_{n \times n} + t \cdot \mathbf{1}_{n \times n} = \begin{bmatrix} a & t & t & \cdots & t \\ t & a & t & \cdots & t \\ t & t & a & \cdots & t \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ t & t & t & \cdots & a \\ \end{bmatrix}.$$

The double-constant matrix has the major eigenvalue $\lambda_* = a-t$ (with multiplicity $n-1$) and the minor eigenvalue $\lambda_{**} = a-t+nt$ (with multipicity one). It can be written in its canonical form (using its eigendecomposition) as:

$$\mathbf{\Sigma} = \lambda_* \cdot \bigg( \mathbf{I}_{n \times n} - \frac{\mathbf{1}_{n \times n}}{n} \bigg) + \lambda_{**} \cdot \frac{\mathbf{1}_{n \times n}}{n},$$

The canonical form presents the double-constant matrix as a combination of the centering matrix and the unit-block matrix. In the case where the double-constant matrix is a covariance matrix it must be non-negative definite, which means that $a \geqslant t \geqslant -a/ (n-1)$. This also means that it will be a scaled version of the equicorrelation matrix (for discussion, see O'Neill 2020, pp. 10, 18-21).


Consider taking a column-vector $\mathbf{x}$ (with the same number of elements $n$ as the rows/columns of the covariance matrix) and using this vector to form the quadratic equation $\mathbf{x}^\text{T} \mathbf{\Sigma} \mathbf{x} = \lambda$. The set of vectors $\mathbf{x} \in \mathbb{R}^n$ that satisfies this quadratic equation forms the surface of an ellipsoid centered at the origin, with orthogonal axes pointing in the directions of the eigenvectors of the double-constant matrix. This result relates to the principal axis theorem which is a theorem uniting geometry and linear algebra.

$\endgroup$
2
$\begingroup$

I wil try to expand on @whuber's comment to the OP. I hope I'm getting this right !

A covariance matrix is expressed as $\Sigma$. It encapsulates how each pair of dimensions in a dataset co-varies. The diagonal elements, $\Sigma_{ii}$, are the variances of each dimension, and the off-diagonal elements, $\Sigma_{ij}$, represent the covariances between different dimensions.

The expression $x^T \Sigma x$, where $x$ is a vector and $\Sigma$ is the covariance matrix, is known as a quadratic form. It is represented as $x^T \Sigma x$ and is a scalar that offers insights into the 'shape' of the data distribution.

The level surface of a quadratic form consists of points where the quadratic form takes a constant value. In mathematical terms, it's the set of points $x$ satisfying $x^T \Sigma x = \lambda$ for some fixed positive value $\lambda$

A covariance matrix is spherical when all variances ($\Sigma_{ii}$) are equal, and all covariances ($\Sigma_{ij}$) are also equal but different from the variances. This structure is denoted as $\Sigma_{ii} = a$ and $\Sigma_{ij} = b$ for $i \neq j$, where $a$ and $b$ are constants. For example:

$$ \Sigma = \begin{bmatrix} a & b & b \\ b & a & b \\ b & b & a \end{bmatrix} $$

or more generally: $$ \Sigma = \begin{bmatrix} a & b & b & \cdots & b \\ b & a & b & \cdots & b \\ b & b & a & \cdots & b \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ b & b & b & \cdots & a \end{bmatrix} $$

For a spherical covariance matrix, the level surface of the quadratic form is a sphere, as for any given value $\lambda$ of the quadratic form, the set of points $x$ satisfying $x^T \Sigma x = \lambda$ forms a sphere. This is where the term "spherical" comes from. In this context it does relate to the geometric idea of a sphere, where all points on the surface are equidistant from the center. In the case of the covariance matrix, it implies that the variability of the data is the same in all directions from the mean (center) but does not automatically imply a uniform distribution of values over this sphere.

$\endgroup$
9
  • $\begingroup$ Could you actually prove the claims in the last paragraph? $\endgroup$ Commented Dec 30, 2023 at 14:48
  • $\begingroup$ @kjetilbhalvorsen I'm not sure but I would try. Perhaps using Mahalanobis Distance ($D = \sqrt{(x - \mu)^T \Sigma^{-1} (x - \mu)}$), substituting $\Sigma^{-1}$ into this, which should reveal that the distribution's density depends solely on the Euclidean distance from the mean so that the distribution is uniform in all directions from the mean, demonstrating spherical symmetry. Thus, a spherical covariance matrix leads to a spherically symmetric multivariate distribution, uniform across all directions from its center. Or are you refering to claim in the last sentence (now not sure about that!) $\endgroup$
    – underflow
    Commented Dec 30, 2023 at 15:46
  • $\begingroup$ you could try the spectral theorem $\endgroup$ Commented Dec 30, 2023 at 15:55
  • $\begingroup$ @kjetilbhalvorsen thanks for the pointer. I'm not familiar with the theorem but after a little reading, I wonder if we could diagonalise $\Sigma$ into $PDP^{-1}$ (where (P) contains orthonormal eigenvectors and (D) is a diagonal matrix of eigenvalues). Here,the off-diagonal elements are equal $\Sigma_{ij} = b$ for $i \neq j$ and diagonal elements are too ($\Sigma_{ii} = a$). Applying the theorem,$\Sigma$ is expressed in terms of its eigenvalues and eigenvectors, indicating equal variances along the principal axes and confirming spherical symmetry.....[cont] $\endgroup$
    – underflow
    Commented Dec 30, 2023 at 16:25
  • $\begingroup$ ....However, I don't think this implies uniform distribution over the sphere, as the distribution pattern depends on the probability density function, not just the covariance structure. Thus, while the Spectral Theorem confirms equal spread, I don't think it addresses the uniformity of values on the sphere. Am I on the right track ? $\endgroup$
    – underflow
    Commented Dec 30, 2023 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.