One other reason is that gradient descent is a more general method. For many machine learning problems the cost function is not convex (e.g., matrix factorization, neural networks) so you cannot use a closed form solution. In those cases gradient descent is used to find some good local optimum points. Or if you want to implement an online version than again you have to use a gradient descent based algorithm.

One other reason is that gradient descent is more of a general method. For many machine learning problems, the cost function is not convex (e.g., matrix factorization, neural networks) so you cannot use a closed form solution. In those cases, gradient descent is used to find some good local optimum points. Or if you want to implement an online version then again you have to use a gradient descent based algorithm.