I think you're confusing the underlying distribution from which both training and test distributions are drawn, with the distributions of the specific train and test draws.
Unless the underlying distribution is eg time-sensitive, changed during the time between eg drawing the training and the testing samples, the underlying distribution is identical each time.
The goal in learning a machine learning model is typically not to learn the training distribution, but to learn the latent underlying distribution, of which the training distribution is only a sample. Of course, you cannot actually see the underlying distribution, but eg, if you only really cared about learning the training samples, you could simply memorize the training samples in a lookup table, end of story. In reality, you are using the training sample as a proxy into the underlying distribution. "Generalization" is a somewhat synonym for "try to learn the underlying distribution, rather than just overfitting to the training samples".
To estimate how well the training data, and your fitted model, matches the underlying distribution, one approach is to draw one training set, one test set. Train on the training set, test on the test set. In reality, since you're most likely fitting a bunch of hyperparameters, you'll overfit these against the test set, think you're getting some super awesome mega accuracy, then fail horribly when you put the model into production.
A better approach is to use cross-fold validation:
- draw a bunch of training data
- split it randomly into 80% training data, 20% valdiation/dev data
- run training/test on this, note down the accuracy etc
- redo the split, eg using a different random seed
- redo eg 5, 10, 20 times, depending on how much variance you are seeing
- this will give you a fairly realistic insight into how well your training sets and model are fitting the underlying distribution
- it's pretty general. You can use this approach for any i.i.d datasets
