I am studying Linear Discriminant Analysis (LDA). According to the formula for LDA, we are supposed to get the inverse of within group covariance. However, if $p\gg n$ (i.e., the dimension is much larger than the number of samples), what should I do?
One possibility is to regularize your estimate of the covariance matrix. See Regularization parameter to generate inverse covariance matrix.
Regularization uses assumed prior-information to turn the covariance inversion into a well-posed problem (this is also mentioned in the wikipedia article)
In practice, you can use a shrinkage estimator of the covariance matrix, such that:
$$\Sigma \rightarrow (\lambda-1)\Sigma+\lambda I_m$$
Where $\lambda$ is a hyperparameter that must be set in advance (and possibly optimized).