7
$\begingroup$

I would like to produce synthetic survey data. At the moment I produce independent answers between questions according to an arbitrary discrete distribution as in this question.

I want to generate randomly and independently answers to 2 different questions with categorical responses.

I want to then generate an answer to a third question which depends on the first two answers.

How can this be done for a continuous valued case?

How can this be done for a categorical case?

I am more interested in how to do the discrete case where a new (dependent) categorical results is produced.

I am interested in any type of dependency which would show up when measuring the mutual information between question answers. Having maybe 2 or 3 category numbers.

$\endgroup$
  • 1
    $\begingroup$ The general approach to inducing dependence would be via copulas, but that may well be overkill for your problem. If you give more details about what you need, something simpler may well be feasible. $\endgroup$ – Glen_b Mar 11 '15 at 14:14
  • 1
    $\begingroup$ The general theory I've already mentioned. It's a subject that fills books; I have several of them on my shelf - could you perhaps ask a question that might be answered in half a page or so? $\endgroup$ – Glen_b Mar 11 '15 at 14:25
  • 1
    $\begingroup$ You seem to misunderstand the scale of what you're asking for. Enough of the general theory can't be usefully conveyed in a paragraph without being facile to the point of uselessness (see the opening 3 paragraphs of the article I linked to). If your primary interest is categorical variables there are additional complications arising from the general theory, so you wouldn't want that in your first example, and even if you stated what sort of dependence structure you wanted, even one example is likely to be getting on the long side if you need it at a basic level, and ... (ctd) $\endgroup$ – Glen_b Mar 11 '15 at 14:45
  • 1
    $\begingroup$ (ctd) ... finally, you're going to have to be much more explicit about what you mean by "test", but my guess is that would also tend to be long for an answer. Ask one specific question please. But I suggest you tackle the wikipedia article first, and then some of the readings it suggests, and then some of the copula questions here, in order to be able to ask a suitable question. The alternative is to drop the insistence on generality and tackle a smaller problem. This is not something that can be conveyed easily in minutes. $\endgroup$ – Glen_b Mar 11 '15 at 14:45
  • 2
    $\begingroup$ One simple (but not remotely general) way to introduce dependence in ordinal categories is with latent variables. You might look at item response theory, for example, as giving a class of models from which dependent categorical data could be generated. $\endgroup$ – Glen_b Mar 11 '15 at 15:01
5
$\begingroup$

Here I use a latent variable approach. This readily extends to the continuous/categorical case.

The idea is to treat a continuous variable (the latent variable) as laying behind the ordered categories that are actually observed (by splitting up the continuous variable at breakpoints).

So for the two variables that are independent, we define breakpoints that give the desired proportions in each category. Then the third continuous variable, correlated with the other two, is also split up in similar fashion. It's common to use standardized normal variables for the latent variables, but other distributions could be used.

The example below is in R but I have annotated it to help conversion to other platforms.

set.seed(10345)    # just to make sure if you run this we have the same results
xu=rnorm(50)       # draw 50 observations from continuous latent variables
yu=rnorm(50)       #
zu= 0.8*xu+0.6*yu  #  the latent variables have correlations 0 between x and y,                
                   #  0.8 between x and z, and 0.6 between y and z

cor(cbind(xu,yu,zu)) # sample correlations will be similar to those population values

px=c(.3,.2,.5)     # our selected population proportions in the marginal categories
py=c(.1,.2,.4,.3)
pz=c(.1,.2,.4,.2,.1)

xc=cut(xu,qnorm(cumsum(c(0,px))),labels=c("AI","AII","AIII")) # convert to ord. categ.
yc=cut(yu,qnorm(cumsum(c(0,py))),labels=LETTERS[1:4])
zc=cut(zu,qnorm(cumsum(c(0,pz))),labels=letters[1:5])

Now let's see the relationships between variables:

enter image description here

table(xc,yc) #examine the resulting data. xc,yc populations are independent
      yc
xc      A  B  C  D
  AI    1  7  9  2
  AII   0  4 11  7
  AIII  2  5 18 14

> table(xc,zc) #xc,zc dependent
      zc
xc      a  b  c  d  e
  AI    4 11  4  0  0
  AII   0  2 19  1  0
  AIII  0  1 18 12  8

> table(yc,zc) #yc,zc dependent
   zc
yc   a  b  c  d  e
  A  1  1  1  0  0
  B  2  7  5  1  1
  C  1  5 27  5  0
  D  0  1  8  7  7

How correlations between the latent variables work.

I chose $X_u$ and $Y_u$ ($u$ for "underlying"; I'd have put $l$ for "latent", but it tends to look like a "1") to be two independent standard normal variates. You can make them correlated with a third variate, $Z_u$, by making $Z_u$ a linear combination of $X_u$, $Y_u$, and an independent noise variate $\epsilon$, which we'll also take to be standard normal here.

If we write $Z^*=aX_u+bY_u+c\epsilon$ then $Z^*$ is normal, but not standard normal.

$\text{Cov}(Z^*,X_u)=\text{Cov}(aX_u+bY_u+c\epsilon,X)=a\,\sigma^2_X=a$

Similarly $\text{Cov}(Z^*,Y_u)=b$ and $\text{Cov}(Z^*,\epsilon)=c$.

$\text{Var}(Z^*)=a^2+b^2+c^2$

So $\text{Cor}(Z^*,X_u)=\frac{a}{\sqrt{a^2+b^2+c^2}}$ and So $\text{Cor}(Z^*,Y_u)=\frac{b}{\sqrt{a^2+b^2+c^2}}$.

But I want $Z_u$ to have variance $1$, so if we define $Z_u=\frac{Z^*}{\sqrt{a^2+b^2+c^2}}$ then $\text{Var}(Z_u)=\frac{a^2+b^2+c^2}{a^2+b^2+c^2}=1$

In the example, I chose $a=0.8,b=0.6,c=0$, which has $a^2+b^2+c^2=1$ and in that case $Z_u=Z^*$, and we have $\text{Cor}(Z_u,X_u)=a=0.8$ and $\text{Cor}(Z_u,Y_u)=b$.

If you choose to have $\text{Cor}(Z_u,X_u)=\rho\,,$ then $-\sqrt{1-\rho^2}\leq\text{Cor}(Z_u,Y_u)\leq \sqrt{1-\rho^2}$ (with the limits being achieved when $c=0$).

Note that these are population correlations, not sample correlations.

In the example you mention in comments, $a=b=\frac{1}{2}$, and $c=0$ which gives $\text{Cor}(Z^*,X_u)=\frac{a}{\sqrt{a^2+b^2+c^2}}=\frac{1/2}{\sqrt{(1/2)^2+(1/2)^2}}=\sqrt{\frac{1}{2}}\approx 0.7071$

-- but now to make $Z_u$ standard normal we need to divide through by

$\sqrt{a^2+b^2+c^2}=\sqrt{\frac{1}{2}}$, i.e.

$Z_u=Z^*/\sqrt{\frac{1}{2}}=\sqrt{2}Z^*$.

$\endgroup$
  • $\begingroup$ (thanks for the answer and making the question better) I am trying to understand the annotation you wrote, "the latent variables have correlations 0 between x and y, 0.8 between x and z, and 0.6 between y and z.* I find a correlation coefficient of 0 between xu and yu, as I expected, and the 0.8 for corrcoef(zu,xu), and 0.6 for corrcoef(zu,yu). How does this arise? With 0.5xu and 0.5yu the corrcoeffs are 0.7 for each. With 1*xu and 2*yu it is 0.45 and 0.9 respectively. Could you explain how the correlation of zu relates to the components in the summation of random numbers? $\endgroup$ – Vass Mar 19 '15 at 18:00
  • 1
    $\begingroup$ I've made some additions describing how the correlations work. $\endgroup$ – Glen_b Mar 19 '15 at 21:14
  • $\begingroup$ thank you for the time and effort you put to answer this question and working with me. I believe that I have learned alot from this. Could you recommend a topic list or reading list to learn more about topics close to this and on how to investigate correlations? I believe that my basics might not be as solid as they should be. $\endgroup$ – Vass Mar 20 '15 at 0:02
  • $\begingroup$ It's very hard for me to judge the right level of book for your purposes; for the correlation calculations, I guess this, this, or this should cover it - or many other things nearby one of them/on the same shelf at a university library. But the information is all on wikipedia ... (ctd) $\endgroup$ – Glen_b Mar 20 '15 at 2:34
  • $\begingroup$ (ctd) ... It's just based off directly applying linearity of covariance, the relationship between covariance and correlation, and some basic properties of variance. Actually I've never seen any reference on the latent variable stuff. I've heard it mentioned once or twice before, and then it's just a matter of figuring out how it works when you need it. $\endgroup$ – Glen_b Mar 20 '15 at 2:40
4
$\begingroup$

Are your variables quantitative or categorical variables ?

In an article we recently wrote, we wanted to simulate three quantitative anwers to a survey : $z$ and $u$ had to be independant, and $X$ had to be correlated to both $z$ and $u$, so we generated them like this :

$\begin{align*} u &\sim \mathcal{U}[a,b] \\ z &\sim \mathcal{U}[a,b] \\ \forall k, X_k &= \alpha \cdot z_k + \beta \cdot u_k + \sigma \cdot \epsilon \\ \end{align*}$

with : $\epsilon \sim \mathcal{N}(0,1)$ and $\alpha, \beta, \sigma \in \mathbb{R}$. I believe it is very common way to proceed, I can think of plenty of papers where people did comparable things.

For categorical variables, I'd suggest a very similar approach :

$\begin{align*} z &\sim \mathcal{B}(n,p)~~~\text{(or whatever distribution suits your problem best)} \\ \forall k, X_k &= \lfloor z_k + \sigma \cdot \epsilon \rfloor \\ \end{align*}$

Parameters $\alpha, \beta, \sigma $ can be fine-tuned to match real survey answers in case you have data at your disposal.

$\endgroup$
  • $\begingroup$ the variables are categorical and quantitative. So you sample the quantitative numbers there from a uniform distribution? For the categorical variables is that the Beta distribution or Binomial? I'm not sure how from either we can get a categorical sample? In the Binomial case maybe by sampling and seeing which side of the distribution we are in to choose between (1or0)? These relationships are linear, is there an easy way to get non-linear relationships? Eg. XOR type relationships? $\endgroup$ – Vass Mar 11 '15 at 14:04
  • $\begingroup$ Actually I assumed you needed to simulate survey data either for Monte-Carlo simulation or for demonstration/teaching purposes. Is it what you're trying to do ? Perhaps are you trying to make thorough checks about theoretical statistics, in which case you might be better off with copulas, just like Glen_b suggested. $\endgroup$ – Antoine R Mar 11 '15 at 19:41
  • 1
    $\begingroup$ In our case, we were doing very general simulations to show how a calibration estimator behaved. We were fine with our quantitative data drawn from a uniform. That might not be the case for you, but same method works for any kind of distribution (say, a log-normal if you're modeling salary). And you can generate non-linear links easily too : for example $X_k = \alpha \cdot e^{z_k}$ As for categorical variables, binomial suited our needs perfectly, but another discrete distribution might suit your needs better (maybe repeated Bernoulli experiments ?). $\endgroup$ – Antoine R Mar 11 '15 at 19:42
  • $\begingroup$ how can the discrete values which are categorical be used in a distribution to sample for a new question? Eg. q1 answer is C, how do we sample a new value for q2 given that q1 is C? Should it be a Bayesian conditional probability? $\endgroup$ – Vass Mar 13 '15 at 16:52
  • $\begingroup$ If you have specified the joint distribution you can certainly calculate the conditional distribution directly, and hence sample that. $\endgroup$ – Glen_b Mar 16 '15 at 3:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.