There is one thing in data science that I cannot understand. When we have algorithms like Random Forest, Gradient Boosting, Neural Networks, we split our data into three parts - train (to train our model), validation (to choose best hyperparameters), and test (to get most accurate out of sample error).
In standard OLS linear regression it's obvious that we won't split our data to training, validation and testing, but only training and testing, since there is no hyperparameters to optimize. However, the game changes when we consider regularization, which implies need of estimation of hyperparameters associated with certain regularization (L1, L2 or Elastic Net). I've never seen in my life, that someone would split data into three parts, to search on the validation set for the best hyperparameters for regularization. In contrast everyone does it on test set.
But this is a little problematic isn't it? Observe, that if you choose regularization hyperparameters on test set, then you don't posses any accurate out of sample result (and this is argument, why you should provide a validation set).
Could you please explain to me this phenomenon?