Monte carlo optimisation (find maximum of function with multiple parameters)

Question

UPDATE 4 UPDATE I JUST NEED TO know name of method(because there are hundreds of mmc methods) I have a description of a Monte Carlo method and don't know if it is a sequential monte -carlo, dynamic monte-carlo? What should I be looking for? Do you know similar methods? This is optimization of functions which have many parameters/Each parameter have range(min max 1-50) F(a,b)=10a+b/b^6 F(a,b) non linear function a range 90-100) b range (2-30)

Monte carlo optimization hav2 parameters Runs And TEST

Runs is sampling with center(so if algorithm found local maximum next samplings will be near this local maximum..

Test are random sampling between all range

Runs

Controls the number of times that the MC optimizer shrinks the parameter windows to re-center them around the resultant closest-to-optimal value.

Test per Run

Controls the number of random tests within each Pass without shrinking or moving the windows.

Assume that you have chosen "Profit" to be maximized. At the start of each pass, the range of random values is decreased and centered on the most profitable value determined in the previous pass. The Monte Carlo Optimizer keeps shrinking the parameter windows as it proceeds, until it winds up centered on a set of values that maximize the targeted metric.

I need to find a description, or articles about Monte Carlo optimsation (find local, global maximum of function). Monte Carlo will generate random inputs and find maximum, then repeat with another parameters). For example,

Input parameter A range [0..1..2..100]
Input parameter B range [0..1..2..30]
F(a,b)=A*sin(A)+cos(b) +104A/B    

F can be any function with a and b parameters so I should find maximum F (and related A,B parameters). Monte Carlo should be in few steps: the first step find a few pairs (A,B) which gave better F(a, b) results) and then the next step should generate random values for a, b (which will be near their range).

It this the standard procedure for Monte Carlo function optimisation? Do you know articles and maybe source code samples for it?

UPDATE: Looks like i i need Weighted Monte Carlo Multi Level Monte Carlo. or another MC model.What can you reccomend? First Monte carlo sampling stage will find A,B combination (for example 5 combiations which give best results and then Next stage of sampling A+1 AND B+! A-1,B-1 ,A+Range B+Range (next monte carlo will make only random sampling inside this ranges) So i need jusn know how this multi step procedure is called? There are many staticians,maybe somebody know,or used before advancedmonte-carlo methods?

UPDATE 2. .I am searching something like this. "the novel multi-level Monte Carlo method (Giles [2006]). The multi-level Monte Carlo method produces an optimal combination of many low-accuracy samples and few high-accuracy samples to reduce computational cost and variance within the model estimator" https://people.maths.ox.ac.uk/gilesm/talks/mcqmc12_giles.pdf

UPDATE 3 FOR @TIM TASK it is time series analysis stock or forex quotes like this so i need to know exactly procedure http://www.zentrader.de/html/monte_carlo_simulator1.html

Answer 1

I think there are two techniques that you may be thinking of

Simulated Annealing and Bayesian Optimization

Answer 2

I think you misconceptualize the Monte Carlo method for finding the global/local maxima.

"Monte Carlo will generate random inputs and find maximum, then repeat with another parameters)": It generates random inputs, but does not find the maximum, just finds the value of the function at that input. Then you repeat this many times with different randomly generated inputs and look at the results to find at which input the function had the highest value. This input will approximately give you the maximum. So the method actually does not know anything about the function, not its gradient or anything, just evaluates the function at given random inputs. Hence it is actually not a maximization procedure in the sense that with every iteration or repetition the algorithm comes closer to the solution.

Now another point to be taken care of is how you generate the random inputs. If your input samples do not cover your whole input range, you will not have looked at the function values at some area of possible inputs and maybe will miss a maximum there, hence you need to cover your whole input space. On the other hand, you need to have enough inputs to make sure you sample your input space densely enough not to miss any local maximum of your function.

I think wikipedia page of Monte Carlo method has good references to start with