# Overfitting using lightGBM?

I have a small dataset composed of 800 data points where I need to perform a regression task. I randomly chose 10% of the dataset to be used as validation.

The problem is that I am not sure if I am overfitting. I can see that RMSE and MAE for the validation dataset is worse than for the training dataset (as expected) but I cannot understand if it is to worse or not.

How can I understand if I am overfitting? How can I solve it?

Define the parameters of the model
params = list(
objective = "regression",
metric = "l1"
)

#Define LightGBM model
model_lgbm_base = lgb.train(
params = params,
nrounds = 50,
data = train_lgbm
)

#Predict
yhat_fit_base = predict(model_lgbm_base, as.matrix(train_model_x[, 2:12])) #Predict in the train data
yhat_predict_base <- predict(model_lgbm_base, as.matrix(val_x[, 2:12])) #Predict in the validation data

#RMSE
rmse_fit_base = RMSE(as.numeric(unlist(train_model_y)), yhat_fit_base) #2.101565 RMSE train
rmse_predict_base = RMSE(as.numeric(unlist(val_y)), yhat_predict_base) #3.329543 RMSE val

#MAE
mae_fit_base = MAE(as.numeric(unlist(train_model_y)), yhat_fit_base) #1.601823 MAE train
mae_predict_base = MAE(as.numeric(unlist(val_y)), yhat_predict_base) #2.384942 MAE val


In addition to the point above and more specific to this post: what we are missing is information about the variability of the performance metric used. For example, if our validation set performance is 80% and our test performance is 78% but we have a variability of $$\pm 6\%$$ in our performance metric, then we do not overfit. If we have variability of $$\pm 0.6\%$$ we do overfit. In order to get some idea of this variability we could use some sort of resampling technique (e.g. $$k$$-fold cross-validation is the simplest form). That is because simply put overfitting is "learning the noise" and resampling allows us to ameliorate noise's influence.