train and validation in a multi model pipeline I have a pipeline which contains two NN blocks one after the other (the second gets as input the first output). I was wondering how to train and validate the two blocks.

*

*Split to train, val and test. run the training on the first block and get the results which will be the input to the second block. the validation phase will be the same.

*I can run train and validation on the first block and the validation of the first will become train+ val for the second block.

*Run k-fold cross-validation on the first block and save the val results for each fold. from this val data we can build a new dataset for the second block.

What is the best work paradigm for this problem?
 A: The point of having separate validation and test sets is to make sure that you verify your results on "unseen" data. If you used the same data, or there were data leaks, you risk having overtly optimistic metrics that could lead to false conclusions about the model performance.
With splitting the data, there are three things to consider. First, how much data is "wasted" on validation vs training ($k$-fold does best in here). Second, how time-consuming or computationally demanding the training would be ($k$-fold is the slowest, as you repeat the process $k$ times, or need to run it in parallel). Third, how reliable the results are (if you verify results only using train set metrics, everything would be fast, no data "wasted", but the metrics would be unreliable). I'd say that an in-between strategy would be to have single training sets used by both models (why not?), separate validation set for each model, and a single test set (there would be only one future, unseen data, you don't need parallel universe for testing the outcomes).
