Here are a few thoughts - hope this helps!
In my experience, I've used cross-validation primarily for assessing internal validity and parameter tuning, so one conventional way to use it is to identify the parameter of interest that you're tuning, optimize it over the 10-folds and re-train the full model on the final selected parameter setting. Once you've trained your final model, I'd hesitate to train further, because you're essentially departing from your "optimal" model setting that you obtained via a cross validation scheme.
For example, say you're applying an $L_1, L_2$ penalty on your model, with associated $\lambda_1, \lambda_2$ parameters dictating its magnitude, then you run the 10-fold CV to find the optimal $\lambda_1^*, \lambda_2^*$, and train the final model with them. If you continue training, then you might end up with different $\lambda^+$ values that are not validated via CV.
If number of epochs is another parameter you want to tune (i.e., want to test 50, 100, 200, etc..), then simply introduce it into your parameter grid search and select the best $n^*$ from your grid search. The one thing to be careful with training a model too long (specially with optimization schemes like gradient descent) is that eventually they'll start to overfit, so check the training and validation error with plots like the below. You basically train until your validation error stops improving (not the training error).
Here's the link to where I got that image (https://www.jeremyjordan.me/evaluating-a-machine-learning-model/). This person has a great tutorial on model evaluation.
I'll defer to others more knowledgeable on memory-optimization strategies - good luck!
