Return to Answer

The problem with $p \geq n$

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \geq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space? (Or if we drew a third blue point, could it fall within one of the larger ellipses?)

If you're a Bayesian, what's your prior on the covariance matrix? A direction perhaps to go is to add some kind of prior or regularization? (I'm admittedly not an expert in this area. I'd love to see other answers.)

And I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?

The problem with $p \geq n$

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \geq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space? (Or if we drew a third blue point, could it fall within one of the larger ellipses?)

If you're a Bayesian, what's your prior on the covariance matrix? A direction perhaps to go is to add some kind of prior or regularization? (I'm admittedly not an expert in this area. I'd love to see other answers.)

And I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \geq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space?

If you're a Bayesian, what's your prior on on the covariance matrix?

I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?

The problem with $p \geq n$

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \geq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space? (Or if we drew a third blue point, could it fall within one of the larger ellipses?)

If you're a Bayesian, what's your prior on the covariance matrix? A direction perhaps to go is to add some kind of prior or regularization? (I'm admittedly not an expert in this area. I'd love to see other answers.)

And I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \leq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space?

If you're a Bayesian, what's your prior on on the covariance matrix?

I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?

If $p \geq n$ then as you recognize your covariance matrix is necessarily rank deficient.

Consider $p=2$ case. If $n=3$ you can find the ellipse defined by the estimated covariance matrix for the three points.

But if $n=2$, then what is the ellipse supposed to be?! The rank deficient covariance matrix defines the degenerate ellipse that's a line segment between the two points. For $p \geq n$, the usual sample covariance has some dimension with zero variation (in this case, basically the $(1,-1)$ vector). Does that mean there's actually zero variation on that dimension, that your data only occupies a one dimensional space?

If you're a Bayesian, what's your prior on on the covariance matrix?

I think we'll have to know more about your problem and why you want the determinant to have an idea of how to proceed? Perhaps you want something like the pseudo-determinant?