How to use the Likelihood Ratio Test and Wald Statistic when also using Cross Validation? I am currently writing a paper for uni and stumbled across the following problem:
I want to use the 10 Fold Cross Validation method to validate the results of a logistic regression, but I am unsure when to use methods like the Likelihood Ratio Test and Wald Statistic in order to also validate the function and coefficients.
Since the coefficients are going to be created through Gradient Descent based on the data and Cross Validation leads to different sub-sets of data for testing and training, the coefficients and results of the Likelihood Ratio Test and Wald Statistic should differ.
So would you normally have an individual approach for each of sub sets of the Cross Validation or would you rather build one model in advance and use it on each of the sub sets?
As you see I am having difficulties getting a hang on the relation between Gradient Descent, Cross Validation and the Likelihood Ratio Test/Wald Statistic, if someone could explain it to me I would be very thankful.
 A: Gradient descent (GD) is an optimization technique used for solving an optimization problem that yields parameter estimates of the model. It is a non-issue here, as GD is used in the same way in all cases. (You would not be using different versions of GD when estimating the model on different folds or the entire dataset.)
Use of the entire sample vs. data splitting by cross validation is an issue. As you note, fitting the same model on different subsets of the dataset will normally yield different parameter estimates.
Since you are fitting a logistic regression and considering use of LR and Wald tests, you must be implicitly assuming your data are i.i.d. (The test results might not be valid if you do not assume that.) To avoid wasting power of the tests, you would run LR and Wald tests on the model estimated on the entire dataset rather than on different (combinations of) folds.
On the other hand, to learn more about the validity of the i.i.d. assumption you could run LR and Wald tests on different (combinations of) folds. Keep in mind that due to the lower power resulting from fewer observations some positive test results may turn negative even if the i.i.d. assumption holds.
