I have two different sets of measures from a group of people who completed various choice tasks: 1) Demographic characteristics of each person. 2) Eye-tracking measures (fixation time, pupil size etc.) of each person.
My goal is to detect whether two different logit models can have different prediction powers. My first logit model (M1) has demographic variables only and my second model (M2) has eye-tracking variables only. So they are non-nested models, but the measures they use come from the same group of people.
I tried to see how these models perform in terms of out-of-sample predictions. So, I run cross-validation (I am not sure about the name of the procedure, but the logic is the same). So, in each iteration I randomly pick 90% of my data ( i.e. I am defining a training sub-sample) and fit my models (M1 and M2 separately) in the training sub-sample , then I am using the obtained fits to predict the responses in the 10% of the data (i.e. in the test sub-sample). I repeat it 1000 times for M1 and M2 independently.
Question: I have two distribution for M1 and M2 from this CV exercise. I expect that distributions are dependent (related) because one is the outcome of a demographic variables based model and the other is the outcome of an eye -tracking measures based model for the same group of people. How do I test that these two distribution are different?