3
$\begingroup$

So, for the purpose of my master thesis I'm trying to predict pfofitabilty on times series data using Elastic net and XGBoost. I split the data 80/20 (50k instances, 3k+ features). I do not use cross validation (tried it, system crashed everytime) or a validation set. I train and tune the model on the training set and evaluate the performance on the test set.

As i have quite a lot of data I was wondering would my simple technique be appropriate or should I try to have an extra validation set for hyper parameter tuning? I would really appreciate any helpful answers or pointers to relevent papers.

$\endgroup$
2
$\begingroup$

It is generally a better practise to use cross-validation (e.g. 10-fold CV) that just a random split to your data. It would be even better if you could use CV and then test your model's performance on a completely independent validation test.You have enough instances to do the later.

Hope this helps.

$\endgroup$
2
$\begingroup$

Gradient boosting has many hyper-parameters and if you have 50k instances I would suggest to use a validation set for hyper-parameters tuning.

"system crashed everytime" is not a valid excuse to not do something important. The solution is just "debug" and try other software packages (if you are using R, caret is a good one to try and it do almost everything for you).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.