There are a variety of ways you can make distributions close (just as there are a variety of ways of estimating parameters from data).
The most common way would probably be to minimize the Kullback–Leibler divergence
$$D_{\mathrm {KL} }(P\|Q)=\sum _{i}P(i)\,\log {\frac {P(i)}{Q(i)}}$$
where $Q(i)$ would represent the probabilities from your known distribution and $P(i)$ would be the one you're "fitting" to it (trying to approximate closely). $P(i)$ is therefore a function of parameters.
I'm going to offer an intuitive motivation for it, along the lines hinted at by your post.
Imagine you generated a really large sample from $Q$, and wanted to estimate the parameters of $P$ by maximum likelihood.
(more in a moment)