I would like to estimate the moments of the joint distribution of continuous and categorical variables in order to draw a synthetic sample from it, and was wondering the correct way to do so.

As an example, if i have vector of continuous variables describing some characteristics of a sample and wanted to estimate their joint distribution i could assume joint normality, and calculate their variance/covariance matrix and means. A random sample could then be drawn from this estimated distribution. If, however, i would like to allow the simulated sample to vary by gender and ethnicity i cannot simply assume normality (not that it's a terrific assumption in the first place) and continue in the same manner i.e by "adding them on" and calculating the new var/covar matrix and mean vector.

Does anyone know of the best way to carry out such a procedure, or of any resources that might help me understand how to do so?


Off the top of my head, you can perhaps use the conditional argument? Let $X$ the binary variable and $Y$ the continuous one then $P(X, Y) = P(Y|X)P(X)$
You can set the marginal probability $\pi=P(X=1)$ and parametrize $P(Y|X)$ e.g. $Y|X \sim N(\alpha + \beta x, \sigma^2)$. Now you have your joint distribution that you can use to draw from.

  • $\begingroup$ I've got a similar problem, but there are several of each kind of variable. Are you suggesting that P(A,B,C,D), where say A,B are continuous and C,D are categorical = P(A,B|C,D)*P(C,D), so I essentially just keep a bunch of (in this example) 2D Gaussians around and select one based on which C,D "coinflip" I get? $\endgroup$ – Pavel Komarov Oct 9 '20 at 17:13
  • $\begingroup$ "Although it is well known that one can use a frequency estimator to obtain consistent nonparametric estimates of the joint PDF in the presence of discrete variables, this frequency-based approach splits the sample into many parts ('cells') and the number of observations lying in each cell may be insufficient to ensure the accurate nonparametric estimation of the PDF of the remaining continuous variables" w4.stern.nyu.edu/ioms/docs/sg/seminars/racine1.pdf $\endgroup$ – Pavel Komarov Oct 9 '20 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy