Suppose I have a family of random variables

$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$

and I know how to sample $SomeDistribution_i$ for any $i$.

Suppose I also define a random variable $Y = f(X_1,...,X_n)$ for some deterministic function $f$.

Then I can approximate the probability of some event $\phi(Y)$ by obtaining samples of $Y$, which in turn is computed from samples of $X_1, ..., X_n$, and checking how often a sample of $Y$ satisfies the conditions for the event $\phi(Y)$.

So far, so good. Now suppose I tell you $SomeDistribution_i$ is the same distribution for any $i$. In other words, $X_i$ are independently and identically distributed. It seems to me this is valuable information that ought to be useful in getting better estimates of $P(\phi(Y))$, because how I have less uncertainty involved (from $n$ unknown distributions to a single one).

However, I don't quite know how to make use of that information. It seems I would still need to generate samples of $Y$ in the same manner as before, without ever making use of the fact that $\{SomeDistribution_i\}_i$ are identical.

In other words, the quality of my approximation of $P(\phi(Y))$ would be exactly the same as the one I obtained when I did not know about the identical distributions.

Is there a way to take advantage of that information? Note that I am not making any assumptions about $f$ or $X_i$, not even whether they are discrete or continuous. Is there a way to use this information even without any assumptions?

One intuition I have is that I could, for each sample of $X_1, ..., X_n$, generate $n!$ samples of $Y$ by permuting $X_1, ..., X_n$ in all possible ways. This would generate many more samples of $Y$ for the same amount of samples from $SomeDistribution_i$. Does that hold water? Is it the best approach? Is there a name for this method?

Edit here's an illustration of the permutation method described. Take for example such an $f$ in $f: \{0,1\}^2 \rightarrow \{0, 1, 2, 3\}$ such that $f(X_1,X_2) = 2X_1 + X_2$ (note that $f$ does not treat both inputs in the same manner). If $X_i \sim Uniform(\{0,1\})$, then $Y=f(X_1,X_2)$ is uniformly distributed over $\{0, 1, 2, 3\}$. If we take a single sample for $X_1,X_2$, say, $(1, 0)$, then the estimated distribution for $Y$ is $P(Y=y) = 0$ if $y=2$ and $0$ otherwise. However, because $X_i$ are identically distributed, the sample $(1,0)$ is just as likely as $(0,1)$ and in the limit they would occur an equal number of times. Therefore it makes sense, given sample $(1,0)$, to deterministically introduce sample $(0,1)$ into the sample set. Such sample set would be more representative of what the sample set will look like in the limit, and provide a better approximation for the distribution of $Y$: $P(Y=y)=0.5$ if $y \in \{1, 2\}$ and $0$ otherwise.

  • 1
    $\begingroup$ No. If you allow $f()$ to be a completely general function of $n$ variables, then I don't believe there is no advantage in knowing that the $X_i$ have identical distributions. The function $f()$ could for example transform each $X_i$ differently, in which case any identity would be destroyed. $\endgroup$ Jul 21, 2021 at 3:59
  • 1
    $\begingroup$ If you have a particular application in mind, perhaps with a more restricted kind of transformation, maybe there is a helpful answer. Also, why are you calling constant $E(Y)$ an "event"? $\endgroup$
    – BruceET
    Jul 21, 2021 at 4:45
  • $\begingroup$ @BruceET, the $E$ was for Event, not (as it may have seemed) Expectation. It could be, for example, $odd(Y)$, meaning $Y$ is an odd number, or $Y > 30$, or any other qualification on the value of $Y$. I will replace the letter to make that clearer. $\endgroup$
    – user118967
    Jul 21, 2021 at 6:04
  • 1
    $\begingroup$ @user118967 You have ascribed to me comments that I did not make. I did not comment on your permutation method at all, although I will do so now. Your permutation method is IMO of limited use, not because of anything about $f()$ but because of the permuted samples are not statistically independent of the original sample and therefore do not contribute as much to the precision of the probability estimate as a new independent sample would do. The best approach will depend on the relative cost of sample generation and function evaluation, issues that you have not discussed. $\endgroup$ Jul 21, 2021 at 22:55
  • 1
    $\begingroup$ "AFAIK there is no specific literature" Yes, that's what I meant. Sorry for typo. "In case f is symmetric, the estimate will be identical, but wouldn't it be helpful to consider sample size much larger, thus providing better CLT bounds?" Absolutely not. The effective sample size is just the original number of $X$ in this case because the permuted $f(X)$ values are perfectly correlated. Permuting makes does not increase the sample size. Treating the permutations as increasing the sample size will give erroneous error bounds. $\endgroup$ Jul 22, 2021 at 5:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.