5
$\begingroup$

I am currently writing a paper for uni and stumbled across the following problem:

I want to use the 10 Fold Cross Validation method to validate the results of a logistic regression, but I am unsure when to use methods like the Likelihood Ratio Test and Wald Statistic in order to also validate the function and coefficients.

Since the coefficients are going to be created through Gradient Descent based on the data and Cross Validation leads to different sub-sets of data for testing and training, the coefficients and results of the Likelihood Ratio Test and Wald Statistic should differ.

So would you normally have an individual approach for each of sub sets of the Cross Validation or would you rather build one model in advance and use it on each of the sub sets?

As you see I am having difficulties getting a hang on the relation between Gradient Descent, Cross Validation and the Likelihood Ratio Test/Wald Statistic, if someone could explain it to me I would be very thankful.

Improve this question
$\endgroup$
1
  • $\begingroup$ What function are you referring to in "also validate the function and coefficients"? $\endgroup$
    – dipetkov
    Jul 25, 2022 at 6:08

1 Answer 1

Reset to default
2
$\begingroup$

Gradient descent (GD) is an optimization technique used for solving an optimization problem that yields parameter estimates of the model. It is a non-issue here, as GD is used in the same way in all cases. (You would not be using different versions of GD when estimating the model on different folds or the entire dataset.)

Use of the entire sample vs. data splitting by cross validation is an issue. As you note, fitting the same model on different subsets of the dataset will normally yield different parameter estimates.

Since you are fitting a logistic regression and considering use of LR and Wald tests, you must be implicitly assuming your data are i.i.d. (The test results might not be valid if you do not assume that.) To avoid wasting power of the tests, you would run LR and Wald tests on the model estimated on the entire dataset rather than on different (combinations of) folds.

On the other hand, to learn more about the validity of the i.i.d. assumption you could run LR and Wald tests on different (combinations of) folds. Keep in mind that due to the lower power resulting from fewer observations some positive test results may turn negative even if the i.i.d. assumption holds.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for you answer, but I am still a little bit confused. In which way/which version would you use GD here? Also when saying "running LR and Wald testson the model estimated on the entire dataset" do you mean 100% percent of the data including the test set (which you normally dont use for training)? And if I would use only one subset (which would be 90% of the data) wouldnt that lead to false result when applying that model on the other subsets? $\endgroup$
    – MauM99
    Jul 25, 2022 at 18:07
  • 1
    $\begingroup$ @MauM99, I do not think the version of GD matters for the question you are asking; you could also replace GD by some other optimization technique. Yes, I think by entire I mean 100%. I would not worry about a false result when applying that model on the other subsets because inference is about the population, not about any particular subset, and the highest-power inference will be obtained by using all the available data. $\endgroup$
    – Richard Hardy
    Jul 26, 2022 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.