# Why is my very simple random forest model still overfitting?

I've been trying to learn some of the basics of machine learning. As a test project, I'm trying to predict NFL wide receivers fantasy football performance by season using data on: their past performance, their team's past performance, and measures of their teammates' ability. My goal is to find a model that predicts player performance better than players' average draft position (ADP) in real fantasy drafts.

I have data from 2000-2019. My full dataset has 1843 observations of almost 200 variables. Note that I use ADP as a variable.

My approach has been to train a random forest model, with cross-validation, on the 2000-2018 data and then measure model performance when predicting on the 2019 data.

The model I used was:

(model0018 <- train(
fantasy_points ~ .,
data0018,
method = "ranger",
metric = "MAE",
tuneLength = 5,
trControl = trainControl(
method = "cv",
number = 5,
verboseIter = TRUE
)
))


The model does not yield good results. My model's predictions only beat ADP 40% of the time for the 2019 data, and that's using ADP as a predictor variable.

When I include 2019 observations in the training set, however, the model predicts these 2019 observations very well. It beats ADP's predictions 84% of the time.

After observing this poor out-of-sample, but strong in-sample performance, I reasoned that my model must be overfitting the data. So I reduced the number of predictors to just 4 predictors I thought would be most relevant.

The model still works very well in-sample, beating the ADP predictions 70% of the time. However, its performance out-of sample was still much worse - beating ADP predictions only 53% of the time.

How is it possible that such a simple model, with 460 observations to every predictive variable, is still overfitting? What can I do to try to fix this problem?