2
$\begingroup$

I have a problem, that sounds very simple in theory but I fail to implement a good solution.

Let my data be a sample of a continuous variable that that follows a normal distribution (m1,v1), associated with some other variables for each point. I want to get a subsample of this data where this variable will follow a different normal distribution (m2,v2). I am interested in the other associated variables, so my aim is just how to subsample my dataset.

m1, v1, m2, v2 are known.

In all the examples, we sample from a normal distribution with mean 10 and target a distribution with mean 8.

Following random subsample with uniform distribution in R which is closer to my question I tried to adapt the method from the answer:

## simulate a sample normally distributed
m1=10
v1=3
N=10000
dist1=rnorm(N,m1,v1)
## define my target distribution
m2=8
v2=3

d=dnorm(dist1,m1,v1)
dist2=dist1[sample(1:N,N,replace=T,dnorm(dist1,m2,v2)*1/d)]
## check    
mean(dist2)
[1] 8.04835
sd(dist2)
[1] 2.982235

And this seems to work ... BUT I noticed we were slightly off, so I performed 10000 such simulations to check

d=dnorm(dist1,m1,v1)
meantot=0
sdtot=0
for(i in 1:10000)
{dist2=dist1[sample(1:N,N,replace=T,dnorm(dist1,m2,v2)*1/d)]
## check
meantot=mean(dist2)+meantot
sdtot=sd(dist2)+sdtot
}
meantot/10000
[1] 8.046662
sdtot/10000
[1] 2.982252

And I repeat this procedure several times, and we stay slightly above the mean. My intuition is that we may fail to have enough numbers before 8 to match exactly the distribution. This small discrepancy could potentially be a problem for my study. Are there any better method for this ? Surely there must have been some work in the past on such problems ?

$\endgroup$
  • 1
    $\begingroup$ I do not understand your problem description. Data does not "follow a distribution". It was sampled from a distribution and the distribution can often be sufficiently described by a standard distribution model. You can sample conditionally from a sample, even in a way that ensures certain properties, but I don't understand how that does relate to "a different normal distribution". $\endgroup$ – Roland Feb 21 '17 at 13:00
  • $\begingroup$ Ok my bad, I must have used the wrong terminology, but I have my sample that happens to fit a normal distribution (m1,v1) and I want to obtain a subsample that happens to follow a normal distribution (m2,v2). I am not equipped to say it in better terms than this, but please help me rephrase if you think that's necessary $\endgroup$ – z_fly_away Feb 21 '17 at 13:09
  • $\begingroup$ I can't help with terminology because I don't get what you are trying to do. $\endgroup$ – Roland Feb 21 '17 at 13:29
0
$\begingroup$

I think that the reason why your dist2 is persistently "off" by some margin is caused by sampling from dist1, which is not exactly normal distributed, but only approximately so.

If you calculate a new dist1 then your dist2 will be again persistently somewhat "off", but by different margins.

Increasing the number of draws N will reduce that margin.

$\endgroup$
  • $\begingroup$ That's a very good point. I have corrected it what I can: taking the mean of the sample instead of the hypothetical mean. Moreover if I increase the N in the example, the sample distribution will be "more normal" and it works. Thanks $\endgroup$ – z_fly_away Feb 21 '17 at 15:03
0
$\begingroup$

if I understand your question correctly, then your solution is trying to "shoot a fly with a canon". The solution to your problem is elementary. If you have a mean of 10 and want a mean of 8, just sample from your original distribution, and subtract 2 from each sampled number - you get a distribution with a mean of 8. If you want to change the standard deviation as well, divide each sampled number (after you subtracted the 2) by the ratio between the old SD and the new SD, and you get a variable with the new SD.

$\endgroup$
  • $\begingroup$ Ok I now realize that was really not obvious from the question but the sample is actually associated with other data, and I want that proportion of the data that would fit into the target distribution. Otherwise yes, this would all be quite obvious $\endgroup$ – z_fly_away Feb 21 '17 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.