1
$\begingroup$

Question: How to weight a sample to remove bias that is seen on many variables?

Background: I've got two samples which are not random. For every individual I've got three categorical characteristics (A, B, C) and one continuous (X). I want to test whether the difference in average X in both samples is statistically significant. Those samples differ in characteristics A, B, C which are known to have influence on X. My idea is to give some weight to every individual in such a way that weighted distribution of characteristics A, B, C in both samples are the same and then compared weighted average. The question is how to calculate the weights? I could define a loss function that tells how different the characteristics in samples are and use some optimisation method to minimize this loss function (maybe stochastic gradient descent).

$\endgroup$
1
$\begingroup$

The classic solution is called direct standardization, see e.g. Chapter 2 of Donald Treiman (2009) Quantitative Data Analysis: Doing Social Research to Test Ideas. San Francisco: Jossey-Bass.

$\endgroup$
  • $\begingroup$ Unfortunatelly I don't have this book, but I've made some googling. Description that I found (metodykv.wz.cz/QDA2_Standardization.ppt) shows how to remove it if we look only by one characteristic. Does this book contain algorithm that deals with multiple characteristics? $\endgroup$ – Tomek Tarczynski Jan 29 '15 at 15:08
  • $\begingroup$ Yes, see example 2 in that chapter. You can also look at other textbooks. I would start with looking at textbooks on demographics or in the biomedical area, as there this technique is still used. $\endgroup$ – Maarten Buis Jan 29 '15 at 15:21
  • $\begingroup$ I've found some example in internet that deals with more than one characteristic. Is the whole idea more or less as follows: Make direct standarization according to interaction of all characteristics? If we have 3 characteristic and each of them have 3 levels then we divide population into 27 stratums and do direct standarization. Is that the idea behind that? $\endgroup$ – Tomek Tarczynski Jan 29 '15 at 15:52
  • $\begingroup$ yes, it is an old technique, so very simple. (Neither characteristic has a negative connotation to me) $\endgroup$ – Maarten Buis Jan 30 '15 at 8:09
  • $\begingroup$ I didn't know this name although it was my first thought. Unfortunatelly there might be problem with number of individuals in some stratums. I would have a case that in sample in stratum I've only 1 observation and in other sample 40 observations. I will try to connect stratums. Thanks for answer. $\endgroup$ – Tomek Tarczynski Jan 30 '15 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.