Suppose we are using rejection sampling and we want to sample from a distribution, say $p$. In order to calculate the acceptance probability, we use the ratio:
$$\Pr\left(u < \frac{p(x)}{Mq(x)}\right) $$ Therefore, we still use the distribution of $p$ for the randomly generated values $x$.
So why do we use rejection sampling? Why don't we sample directly from $p$? In other words, what makes a distribution difficult to sample from?
Therefore we still use the distribution of p for the randomly generated values x.
Computing the density, $p$, isn't the same as sampling random variables from the distribution $p$ is the density for.
Why don't we sample directly from p?
I'm uncertain what you mean by "sample directly". How do you propose to sample directly from $p$ in general?
Consider the following density, for example: $f_X(x) = c.\exp(-\sqrt{1+x^2})$ (to my recollection, $c$ can be computed in terms of Bessel functions, but its value is unimportant right now). Maybe you can figure out how to sample from that "directly" (whatever you mean by that) if you're sufficiently clever ... but personally, I'm usually not quite that clever -- and generally I'd just use (a variant of) rejection sampling for something like that.
[One additional advantage of rejection sampling in this example is that I don't even need to know $c$ to use it. There are some simple proposal densities that will work nicely for this case.]
In other words, what makes a distribution difficult to sample from?
That depends on what tools you have for generating random variables. If you know many tools, some distributions are not difficult. If you only have a few tools, many more things become difficult to sample from.
(For example, if you only know how to generate random numbers by use of the probability integral transform, then any density whose inverse cdf is difficult to evaluate will be difficult to sample from. Even if it's not difficult, it may not be computationally fast - any distribution whose inverse cdf is expensive to evaluate will be expensive to sample from.
You might like to consider a Tweedie distribution with $p$ between 1.7 and 1.8, say (where you only need a few observations at any one value of $p$). It has some point mass at 0, but that can be dealt with. Nevertheless, the inverse cdf for the continuous part is problematic, to say the least. Even the density involves an infinite sum (which, nonetheless, can be evaluated, though it's expensive to calculate). Rejection sampling - with some tweaks to reduce function evaluations to a minimum - is possible here. Inverse cdf? I don't know that it would be practical, even if it were possible. You might figure out a way to do it using the inverse cdf if you're really clever - I think it might be beyond me, but even if you do, it's going to take a while to get your numbers.)
Rejection sampling is a very important tool in that collection (or rather, rejection sampling is itself a class of tools, because there are many clever variations on the idea), perhaps one of the most important tools. Variations on it are very widely used.
So why do we use rejection sampling exactly?
There are a few reasons to use rejection sampling. Historically, computation was the bottleneck in random variate generation. Calculation of functions like $\log(\cdot)$, $\exp(\cdot)$ and $\sin(\cdot$) were time consuming. Some rejection techniques enabled avoiding the computation of these functions entirely.
Presently (as of 2022), one common reason is that sampling from a distribution (say ($p(\cdot)$) itself is difficult, whereas sampling from $q(\cdot)$ is easier. Let's call sampling from $p(\cdot)$ by the inverse CDF method the iCDF method for brevity. This reason is perhaps less likely to change with time.
Another reason to use rejection sampling is efficiency. For example an iCDF method could be $\mathcal{O}(\theta)$ where $\theta$ is a parameter of the distribution $p(\cdot)$ whereas a rejection method might be $\mathcal{O}(1)$ so that-in theory anyway-the rejection method can be used by default in a sampling library. We'd use it as a default because you wouldn't know in advance which of the particular values of $\theta$ a user of the library would need for a particular bit of code. In practice you'd usually find some cutoff or threshold-if one exists-in the $\theta$ space and use rejection sampling in the $\theta$ region where more efficient and iCDF where the latter is more efficient.
Why don't we sample directly from $p(\cdot)$?
Some times it may not be possible to use the iCDF method to sample directly (this is how I'm interpreting the word 'directly'). An example is the Kolmogorov-Smirnov distribution. As far as I'm aware the only methods to sample this distribution either use numerical approximations or they use a variant of rejection sampling known as the Series method. This is an example where it isn't possible to sample directly unless you want to accept numerical tradeoffs which add a lot to library maintenance and implementation overhead. Also see my comment about efficiency in the answer to the first question which also applies to this question, that is another reason to use rejection sampling.
In addition there are many variants on rejection sampling, like Vaduva's method, the series method, and the Ratio-of-Uniforms method. You can read about each of them in Luc Devroye's wonderful book Non-uniform Random Variate Generation.
In other words, what makes a distribution difficult to sample from?
This is perhaps too broad of a question to answer. To paraphrase Tolstoy,
All easy distributions are alike, but every difficult distribution is unhappy in its own way.
What makes a distribution easy to sample from is light tails and a finite mode. Light tails and a finite mode imply that rejection sampling can be used with a suitably dominating distribution. Various tricks and techniques may be required to recognize an easy form but they all seem to follow this pattern.
Sampling from a distribution means getting a sample with its probability matching the pdf of that distribution. In rejection sampling, we take random samples and select such that the pdf of these samples donot exceed the pdf of given distribution. This is the rejection criteria. This way we have appropriate samples and by getting large number of random sample under this criteria will represent the total picture of distribution.
