Consider the high dimensional data with which the number of features $p$ is much larger than the number of observations $n$. Machine learning algorithm is trained with the data.
My first thought is that a learning algorithm trained with the high dimensional data would have large model variance and so poor prediction accuracy.
To construct a model, we need to decide the parameters of models and the number of parameters gets larger when the number of features increases. And for the wide data, we would not have enough observations to decide all the parameters reliably. I think that the parameters of the model will change sensitively with the change of train samples. The instability of the model parameters indicates that there would be large model variance which will worsen the prediction performance.
However, I read that the machine learning models trained with high-dimensional data can make good predictions. I am curious about what is the underlying reason ML works for the prediction of the high-dimensional data($n \ll p$).