I am doing the working assumption that by "whitening", we mean "sphereing" in the sense that data is uncorrelated and of equal variance along each dimension (not just to remove the mean and rescale the variance to 1 per dimension). I also note that the even when we are in $n > p$ setting the whitening matrix itself is non-unique; the three main ones are PCA whitening, ZCA whitening (sometimes called Mahalanobis whitening) and Cholesky whitening. The PCA is likely the most commonly used but the other are equally correct.
While anti-climatic I don't think there is a simple solution to this when $p > n$, $n$ being the number of observations and $p$ being the number of variables/dimensions/coordinates. That is because the information for the rescaling all the variables while accounting for the covariances simply isn't there.
The simplest sphering can be done via the Cholesky decomposition, we effectively take the covariance matrix $C$, invert it to get $C^{-1}$, decomposite as $C^{-1}=LL^T$ and use $L^T$ as the "sphering matrix". In the case of $C$ being rank-deficient though, the inverse doesn't exist so our immediate avenue is stopped. As suggested we can try using the Moore–Penrose pseudo-inverse to get our pseudo-inverse $C^{+}$ such that $CC^{+}C=C$. That is unfortunately a dead-end because the Cholesky decomposition still needs a positive definite (PD) matrix (we can actually get some little extra mileage out of the $LDL^T$ decomposition as it is willing to play ball with positive semi-definite (PSD) matrices but as soon as $p - n> 1$ things break down again). The pseudo-inverse $C^{+}$ is not PD so Cholesky whitening is a no-go.
Next we try ZCA; ZCA actually has two equivalent formulations, one based on the inverse of the covariance matrix and one based on SVD ($USV^T = X$), the sphering matrices for those are $C^{-\frac{1}{2}}$ (that's why it is called Mahalanobis) and $VS^{-1}V^T$ respectively. The case of $C^{-\frac{1}{2}}$ is a no-go as described above - even if we get the inverse getting the square root of it is impossible as that matrix is not PD. The SVD one is the more promising as we do not invert anything directly but then we are faced with another problem: despite it's best intentions, the diagonal matrix $S$ is $n \times n$, not $p \times p$. That means that simply put, we cannot rescale $p$ varibles with $n$ variances if $n < p$ ($V$ will be $n\times p$ so that's fine). So that again doesn't work.
Finally we have the PCA variant; it uses the eigendecomposition of the covarinace matrix $C = U\Lambda U^T$ and the sphrering matrix is simply $\Lambda^{-\frac{1}{2}}U^T$. Now, if $C$ is rank-deficient we will still get $p$ eigenvalues which is a step forward from the SVD above but those eigen-values are numerically zero as $C$ is (at best) PSD. Effectively the eigendecomposition comes back deficient by itself. Granted we can set small/numerically zero eigenvalues to something like $\epsilon$ and get the whole pipeline working but the resulting sphrered data will be have $p-n$ columns with near $\epsilon$ variance and likely near $\epsilon$ covariance with the other columns. The reason for that is that by "fixing" the numerically zero eigenvalues we zero-ed the variance along these components/coordinates.
To recap, we cannot directly whiten/make spherical a rank-deficient matrix using any of the standard sphrering methodologies. Some methodologies do not compute at all (e.g. Cholesky whitening) and some truncate the dimensions that have variabillity (e.g. PCA whitening). I used Optimal whitening and decorrelation by Kessy et al. (2018) as my standard reference. Do note that we can still using ridge or LASSO regression if we want to use that data further, we just need to standardise the columns in that case which is by itself much easier to do.
