3
$\begingroup$

Background

I have a population in which my dependent variable is binary with a highly-skewed distribution: Very few records are 1 (doers), most records are 0 (non-doers). I'm using logistic regression to predict the value of this variable.

To deal with this rare-event situation, I made multiple samples of non-doers that are size-matched to the number of doers (e.g., sample 1 has the 100 doers and 100 non-doers, sample 2 has the 100 doers and a different set of 100 non-doers, etc.).

I fit a logistic regression to each of these samples and computed t-values to quantify the degree to which the independent variable coefficients are reliably different from 0. That is, each independent variable is associated with1 t-value per sample.

Question

Can I compute statistics of t-values? Can I compute the average, standard error, and standard deviation of these t-values? I don't think I can do it for p-values and I know that correlation coefficients need to be transformed to z-values first, but I don't know what's acceptable for t-values.

$\endgroup$

1 Answer 1

1
$\begingroup$

Normally you wouldn't work this way.

There's really no additional information from splitting your larger sample up in such a way... so I think an important question is 'what do you think you're getting by doing this?'

What information are you trying to get by averaging t-values?

There is actually at least one situation where you might summarize information from many p-values, even average them -- and in that case you might also use t-values - whether averaged or in some other way. In general I think you're better to average things like effect sizes ... but if you're going to do any of these things you have to take account of the dependence between samples that's in your scheme ... and if you do that properly, what do you gain over just using all the data to being with?

$\endgroup$
1
  • $\begingroup$ Thanks for your response, Glen_b. I'm splitting my larger sample to avoid training a classifier on data with a dependent variable that has a highly-skewed value distribution. Also, because this way I can limit the amount of data that I have in memory at any given time. By averaging values, I'm trying to determine whether the t-statistics across samples are reliably different than zero. What is the situation in which you can summarize information from many p-values? Why are effect sizes preferable? How do I take into account the dependence between my samples? $\endgroup$
    – Gyan Veda
    Mar 17, 2014 at 19:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.