We know that the closed-form solution for linear regression is $\beta = (X'X)^{-1}X'Y$.
$X$ is a $N\times M$ matrix, where N is the number of observations and M is the number of features.
However, in the case when we have more features than observations, $M>N$, $X'X$ is not longer invertible since the $M\times M$ matrix of $X'X$ is not full rank. As a result, we can no longer use the closed form solution.
Would it be possible (and reasonable) then to use the gradient descent approach to solve for $\beta$?
It is commonly mentioned that gradient descent is preferred to using the closed-form solution since it doesn't require inverting the $X'X$ matrix which can be time intensive, especially for large $M$.
I was curious if gradient descent can also always guarantee a solution when the closed-form solution can't.