Generating random variable from no closed-form marginal density [closed]

Question

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int_u f(y|u)f(u)du$.

$x(t)$ is composite function of $u$, basically $x(t)$ is a function of $z(t)$ and $z(t)$ is a function of $u$. I can provide the expression of $x(t)$ in terms of $u$, it's too complicated(involves exponent terms, integrand in dinominator), but I feel it's probably unnecessary. Another info is that the marginal pdf of $y$ has no-closed form expression.

I think what I need to do is generate random variable from $f(y)$, I am reading Monte Carlo method textbook, but there is no detail or probably I am newbie about generating random variable. The textbook is giving an example of normal mixture and the example is given Generating random variables from a mixture of Normal distributions here too, but it's not similar what I am dealing with.

I have also seen this how to generate data from cdf which is not in closed form? and Multivariate and Marginal simulations, but still so confused. The fact is how do I assume the distribution(marginal distribution) of $y$. Or do I need to go for indirect method (eg. accept-reject method)?

Like in the last link they are telling to generate $y_1^{(1)}$ from $f(y_1)$ first. But how do I do that if I don't know the $f(y_1)$

I want to implement the algorithm in r, but I am stuck at the very beginning of making the algorithm.

Answer 1

A marginal distribution on $Y$ associated with a joint distribution on $(U,Y)$ is by definition the distribution of $Y$ on its own, that is, ignoring the realisation of $U$.

As an example, consider the joint density $$f(u,y)=\varphi(u;0,1)\times\varphi(y;u,1)$$ for which

1. $Y|U=u$ is conditionally distributed as a $\mathcal N(u,1)$
2. $Y$ is marginally distributed as a $\mathcal N(0,2)$ variate.

and both statements are simultaneously correct. This implies that if one jointly simulates $(U,Y)$ from the joint Normal distribution, the resulting $Y$ will be distributed from $\mathcal N(0,2)$. As an illustration, here are 100 $\mathcal N(u_i,1)$ densities when $$U_1,\ldots,U_{100}\stackrel{\text{iid}}{\sim} \mathcal N(0,1)$$ and their average density (in red):

which is very close to a $\mathcal N(0,2)$ density.

A mistake in considering that since $Y\sim \mathcal N(U,1)$ it cannot be $\mathcal N(0,2)$ at the same time is to forget that $U$ is a random variable. (There is some similarity with accept-reject in that, while all simulations are from the proposal distribution $g$, those that are accepted are also simulations from the target distribution $f$.)

Another illustration is provided by finite mixtures of distributions: if $$Y\sim f(y)=\sum_{i=1}^k\omega_i f_i(y)\qquad\sum_{i=1}^k\omega_i=1$$ this distribution can be represented as the marginal distribution (in $Y$) of a joint distribution on $(U,Y)$ such that $U$ is marginally distributed as a Multinomial variable: $$\mathbb P(U=i)=\omega_i \qquad i=1,\ldots,k$$ and the conditional distribution of $Y$ given $U$ is $$Y|U=i \sim f_i(y)$$ Generating an index $i$ with probability $\omega_i$ and then a realisation from $f_i(\cdot)$ is equivalent to generating from $f(\cdot)$.