I want to find the parameters of a model which specifies a set of classification probabilities, for say M classes. (I'll use the parameters in another model later.)

Given a set of parameters $\theta$, I can calculate its implied classification probabilities, $\pi(\theta)$, using Monte Carlo integration - $\pi$ is a smooth vector function of $\theta$. I would keep changing the parameters until I get the $\theta(\mathbf{p})$ which generates the classification probabilities I'm looking for - fixed $\mathbf{p}$. (I'll set the seed to control the randomness.)

So, I want to use Nelder-Mead to minimise the "distance" between the $\mathbf{\pi(\theta)}$ and $\mathbf{p}$. I would like the distance to be Kullback-Leibler or Kolmogorov-Smirnoff as opposed to least squares for my fitness function.

Which fitness function will lead to the solution faster - KS or KL? Also, given that I can evaluate the function to be optimized, but not its derivatives, is Nelder-Mead my best option.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.