I a have a series of small samples taken not randomly from a reference population with a complex distribution over a categoric variable with 21 levels and a continuous one in the x > 0 domain.

I would like to check which of these samples is more representative of the reference distribution. My approach until now has been to divide the continuous variable into 20 groups, compute the frequency of the combination of the two variables (now both categorical) and compare it with the same combinations in the samples. For the comparison, I thought of using either Cramer V or Spearman correlation or the multiplication of both.

Does it make sense? which of the three correlation methods (cramer, spearman, multiplication of both) should I use?

Another nicer possibility would be to use the posterior probability of the population distribution given the sample, using the second answer to this: Probability that a sample came from a known distribution, but I'm not sure how to build the prior (should I just set it to 1?).

Finally, these methods are based on the arbitrary subdivision of the continuous variable in 20 groups. It would be better to able to use the variable as continuous but I wouldn't know how.

  • $\begingroup$ As Tim answered in the link you included, you can use log likelihood to determine how likely the sample is given a true reference, this works for any distribution. $\endgroup$ – user2974951 Sep 11 '19 at 13:39
  • $\begingroup$ Just the likelyhood and not the full posterior p(M|y)? What about a solution to avoid the categorization of the continuous variable? $\endgroup$ – Bakaburg Sep 11 '19 at 14:15
  • $\begingroup$ The frequentist approach does not require a prior (and there is no posterior). I do not know what exactly you are doing with the categorization, but likelihood does not require it. $\endgroup$ – user2974951 Sep 11 '19 at 14:20
  • $\begingroup$ I use categorization because I don't know the distribution form of the continuous variable given each level of the categorical one. So discretizing it helps me create an empirical probability distribution along the two variables; then I simply need to assign the probability to the data points. $\endgroup$ – Bakaburg Sep 11 '19 at 14:30
  • $\begingroup$ What do you mean by "representative"? The correlation is just a number, which in an overwhelmingly large number of cases is a number with little to no meaning. If you consider how the correlation is computed, you only get a seriously reduced piece of information. $\endgroup$ – cherub Sep 12 '19 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.