5
$\begingroup$

I want to find the parameters of a model which specifies a set of classification probabilities, for say M classes. (I'll use the parameters in another model later.)

Given a set of parameters $\theta$, I can calculate its implied classification probabilities, $\pi(\theta)$, using Monte Carlo integration - $\pi$ is a smooth vector function of $\theta$. I would keep changing the parameters until I get the $\theta(\mathbf{p})$ which generates the classification probabilities I'm looking for - fixed $\mathbf{p}$. (I'll set the seed to control the randomness.)

So, I want to use Nelder-Mead to minimise the "distance" between the $\mathbf{\pi(\theta)}$ and $\mathbf{p}$. I would like the distance to be Kullback-Leibler or Kolmogorov-Smirnoff as opposed to least squares for my fitness function.

Which fitness function will lead to the solution faster - KS or KL? Also, given that I can evaluate the function to be optimized, but not its derivatives, is Nelder-Mead my best option.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.