I want to find the parameters of a model which specifies a set of classification probabilities, for say M classes. (I'll use the parameters in another model later.)
Given a set of parameters $\theta$, I can calculate its implied classification probabilities, $\pi(\theta)$, using Monte Carlo integration - $\pi$ is a smooth vector function of $\theta$. I would keep changing the parameters until I get the $\theta(\mathbf{p})$ which generates the classification probabilities I'm looking for - fixed $\mathbf{p}$. (I'll set the seed to control the randomness.)
So, I want to use Nelder-Mead to minimise the "distance" between the $\mathbf{\pi(\theta)}$ and $\mathbf{p}$. I would like the distance to be Kullback-Leibler or Kolmogorov-Smirnoff as opposed to least squares for my fitness function.
Which fitness function will lead to the solution faster - KS or KL? Also, given that I can evaluate the function to be optimized, but not its derivatives, is Nelder-Mead my best option.
