Suppose I have a dataset $\mathbf{X}$ which is a $n \times m$ matrix of $n$ independent realizations of some $m$-dimensional random vector $\mathbf{x}$. I want to generate a new dataset $\mathbf{X}'$ of size $n' \times m$ such that $n' \gg n$ and $\mathbf{X}'$ has the same first, second, and third moments $\mathbf x$; such moments are not known a priori are are be estimated from the matrix $\mathbf X$.

If only the first and second moments were required, I know that there is a very fast and easy trick for doing this this with a Cholesky decomposition which avoids having to construct and sample a multivariate distribution from the estimated moments. I will describe this trick here, because I am looking for an analogous trick with incorporates the third moments as well.

First, assume w.l.o.g. that the mean vector of X is zero, we can always trivially add that back in. For the second order moments, take the Cholesky decomposition of the covariance matrix:

$$ \mathbf L \mathbf L^T = \mathbf C = \frac{\mathbf X^T \mathbf X}{n-1} \tag{1}$$

Then we could generate a matrix $\mathbf Z$ by sampling each element from a standard normal distribution.

$$ \mathbf X' = \mathbf Z \mathbf L^T \tag{2} $$

is the required matrix. This works because the emperical covariance matrix of $\mathbf Z$ converges to $I$ by construction, so we can prove $\mathbf Z \mathbf L^T$ will have roughly the same empirical covariance matrix as $\mathbf X$:

$$ \begin{align} \frac{(\mathbf Z \mathbf L^T)^T(\mathbf Z \mathbf L^T)}{n-1} = \frac{ \mathbf L^{} \mathbf Z^T \, \mathbf Z^{} \mathbf L^T}{n-1} = \mathbf L \frac{ \mathbf Z^T \mathbf Z }{n-1} \mathbf L^T \xrightarrow{n'\to\infty} \mathbf L^{} \mathbf I^{} \mathbf L^T = \mathbf L^{} \mathbf L^T = \mathbf C \end{align} \tag{3} $$

So that's how the Cholesky trick allows us to generate random data with the same first and second moments as some seed dataset. However, it completely ignores any higher moments. However, the data I'm interested in clearly has several very skewed variables. Is there a method that improves upon (2) so that the empirical skewness and coskewness of the generated data $\mathbf{X}'$ is that same as $\mathbf X$?

  • $\begingroup$ Several elements of your question suggest there may be some need to clarify it. Do you specify $n^\prime$ or can that be part of the solution? Are there any other restrictions on $X^\prime$? (If not, what is $X$ needed for except to specify the moments you want to reproduce?) Is there a reason you are dividing by $n-1$ instead of $n$ as required for the covariance matrix? What is the point of generating random matrices in order to solve a deterministic mathematical problem--is there something you are quietly assuming that needs to be made explicit? $\endgroup$ – whuber Jul 12 '19 at 2:47
  • $\begingroup$ 1) I specify $n'$ and in the intended application it will be 10 times $n$. 2) $\mathbf X$ is only used to obtain the moments, yes. The problem could be restated without mentioning $\mathbf X$ at all: given the first three moments of a multivariate distribution (a mean vector $\mathbf \mu$, covariance matrix $\mathbf \Sigma$, and coskewness tensor $\mathbf K$, generate $n'$ realizations such that the empirical estimates of mean, covariance, and coskewness of the generated dataset converge to the given. $\endgroup$ – olooney Jul 12 '19 at 12:09
  • $\begingroup$ 3) I'm dividing by $n-1$ because I think that is the formula for a sample covariance matrix. Wikipedia describes this as Bessel's correction and is intended to make the estimate unbiased. I note I used the notation $\operatorname{Cov}[\mathbf X'}$ which is incorrect and makes $\mathbf X'$ look like a random variable; I'll change that. $\endgroup$ – olooney Jul 12 '19 at 12:11
  • $\begingroup$ Thank you for the clarifications. Because they differ sharply from how you have asked the question--your post asks for a dataset $X^\prime$ rather than a distribution--please edit the post to help readers understand what kind of an answer you are looking for. $\endgroup$ – whuber Jul 12 '19 at 12:14
  • $\begingroup$ 4) The problem is given as "generate a dataset" $\mathbf X'$ in analogy to the Cholesky trick - which turns out to be a very fast way to generate lots of data, and as far I can see works without explicitly calculating or using a distribution for the data. If such a shortcut is known for higher moments it would probably be the best way to solve my problem, which is to generate a lot of data that looks vaguely like an existing data set for testing purposes. $\endgroup$ – olooney Jul 12 '19 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.