I am running repeated K-fold Cross-Validation on my dataset using different models. My problem is a regression problem and I am counting on the error metric MAE. I do know that some models may behave better that others, but I cannot explain the drastic change in the behavior of the learning curves in each:
Specifically, I want to know what makes the learning curve for the gradient boost so bad. I assume its not learning anything. Is that normal ?
It seems that boosting algo is overfitting because train error is around 0 constantly. You could try to reduce cimplexity of the model e.g. try to use smaller tree depth or some other stopping criteria.
Your boosting algo is underfitting as it has high variance (testing mae is high). This means your model is not learning from the data. There might be 2 reasons for this. The dataset is too complex for your model or your model is too simple and cannot learn the complexities of the dataset. Possible solutions might be:-
1.) Increase the size or number of parameters in the ML model.
2.) Increase the complexity or type of the model.
3.) Increasing the training time until cost function in ML is minimized.
