For the training of a machine learning model I need to add additional features, and these features are correlated. I need to run the model N times adding these features with random values, and for this I use Cholesky decomposition.

Now, I need the user to define these features in ranges, for example:

Feature 1 between 1000 and 2000
Feature 2 between 3 and 5
Features 3 between 20 and 30

With the Cholesky decomposition I can take one of the features and get the others.

But what I need is the system to calculate randomly N sets of random features, within the ranges. So the input is not one of the features, but all of them.

If the ranges provided don't make sense (i.e. there's no way to use the correlation to get a set within the ranges), then the process should fail.

Is there a way to accomplish this (either with or without Cholesky)?

  • $\begingroup$ I presume by "Cholesky" you mean you are using the Chloeslky factor of the covariance matrix of a Multivariate Normal to generate (correlated) random draws from a Multivariate Normal distribution having specified mean and covariance matrix. If you wish random draws to be generated per a probability distribution for which the random variables are restricted to finite ranges, that would not be Multivariate Normal, and there are an infinite number of possibilities for what distribution to use. Perhaps some type of copula en.wikipedia.org/wiki/Copula_(probability_theory) would make sense. $\endgroup$ – Mark L. Stone Jun 16 '19 at 12:06
  • $\begingroup$ I'm not sure I understand what you try to do, but if you just want random features with given correlations and given ranges, one shot would be to just use normal distributions with the mean in the middle of the range and a standard deviation that is something like range divided by 10. You can specify the covariance matrix you want to give you the desired correlations. And then you truncate whatever points are outside the range. The probability for such points is very, very low, so your correlations will not exactly match the specified ones but at very high precision. $\endgroup$ – Lewian Jun 16 '19 at 12:19
  • $\begingroup$ PS: There may be some literature about the correlations of truncated normals given the covariance matrix of the underlying normals, which could help you to get a precise solution based on my earlier proposal. $\endgroup$ – Lewian Jun 16 '19 at 12:21
  • $\begingroup$ so, if I understand correctly, then each of the new features needs to be a) correlated to all the existing features and b) within a specific range. Is that correct? $\endgroup$ – Dave Jun 16 '19 at 14:36
  • $\begingroup$ @Dave, there are two type of features, some are correlated (the ones that I need to generate randomly within the provided ranges). The additional features need to be generated. $\endgroup$ – ps0604 Jun 16 '19 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.