There are a variety of ways you can make distributions close (just as there are a variety of ways of estimating parameters from data).

The most common way would probably be to minimize the [Kullback–Leibler divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence)

$$D_{\mathrm {KL} }(P\|Q)=\sum _{i}P(i)\,\log {\frac {P(i)}{Q(i)}}$$

where $Q(i)$ would represent the probabilities from your known distribution and $P(i)$ would be the one you're "fitting" to it (trying to approximate closely). $P(i)$ is therefore a function of parameters.

I'm going to offer an intuitive motivation for it, along the lines hinted at by your post.

Imagine you generated a really large sample from $Q$, and wanted to estimate the parameters of $P$ by maximum likelihood.

(more in a moment)