How to evaluate how well some samples come from some given distributions? I have a complicated procedure that, given an input $x$, outputs several random samples $S_i(x)$ for $i\in\{1,\dots,k\}$ (each sample consists of $n_i$ points). Each sample $S_i(x)$ follows an unknown distribution $F_i$. I wish to find $x^*$ that will make my samples $S_i(x^*)$ "look as close as possible to" a set of predefined Normal Random Variables $F^*_i =\mathcal{N}(\mu_i, \sigma_i)$. How can I measure this "looking as close as possible to some distribution"? I also wish that this metric weighs equally each $F^*_i$ and that this weighting does not depend on the parameters of $F^*_i$ ($\mu_i$ and $\sigma_i$).
Initially I thought of simply calculating the likelihood of the sample, though this would work with discrete distributions, it wouldn't for continuous distributions as the likelihood of a sample is always $0$ (edit: See @amoeba answer as on why this is false).
 A: 
I thought of calculating the likelihood of the sample, but this would work with discrete distributions. But for continuous distributions the likelihood of a sample is always $0$.

This is a misconception.
If you have a point $x_i$ and a probability distribution $f(x)$, then the likelihood that this point comes from this distribution is defined as $f(x_i)$. If $f(x)$ is the PDF of a normal distribution, then $f(x)\ne 0$ for any $x$.
If you have an iid sample $S=\{x_1, x_2, \ldots, x_n\}$, then the likelihood of it coming from the same distribution is $\prod_{i=1}^nf(x_i).$
I think this should be enough to get you going. Note that if $f(\cdot)$ is normalized to integrate to $1$, then values $f(x_i)$ tend to be small and when multiplied together quickly become close zero. That is why we are usually working with log-likelihood instead of likelihood: $$\log\Big(\prod_{i=1}^nf(x_i)\Big)=\sum_{i=1}^n\log f(x_i).$$

Having said all that, it seems that in your case you should compute some goodness-of-fit statistic and not the likelihood. The difference is illustrated in the first comment by @BrentKerby.
A: The goodness of fit of $S_x$ to $X^*$ can be assessed in a number of ways: the Cramér-von Mises statistic, the Kolmogorov-Smirnov statistic, or chi-square statistic would be common choices, and there are certainly others. If we knew something about the distribution of $S_x$, or about your specific goals or performance criteria, then it might be possible to say more.
