I have a physical model which takes $\sim50$ parameters and gives $\sim 2000$ outputs taking tens of minutes to run. I need to optimise these parameters to give outputs as close as possible to data by minimising the $\chi^2$. The problem of course is that it is expensive to evaluate and, probably worse, that there are so many parameters. Nelder-Mead for example fails to converge in an acceptable number of iterations.
I know very similar questions have been asked before but the key difference in this case is that it is quite intuitive to fit to the data to some extent by hand. This means the parameters are already reasonably close to optimal from the start, so even a local optimum could be fine.
Would this fact alter the approach? Many of the answers in Optimization when Cost Function Slow to Evaluate for example suggest Bayesian optimisation, but this assumes a black box function and the stochastic nature feels somewhat wasteful. Maybe I am wrong however and it would actually still be a good approach.
So, is there maybe an approach more suitable for this case (when one is already close to the optimal point and there are a large number of parameters)? Perhaps some kind of machine learning algorithm would be able to learn how certain parameters affect the output in a similar way to how it can be done by hand and hence more efficiently optimise it to the data, or maybe Bayesian optimisation would in fact be fine if somehow limited to the region around the optimal point?
