# Numerical method to minimize variance of single-parameter black-box distribution

I am looking for suggestions for numerical methods to solve the following problem on a computer:

I have a single-parameter distribution, from which I can generate individual samples. Consider the distribution a black box. I have no closed-form formula for it. I am looking to find the parameter value which minimizes the variance of the distribution.

To goal is to do this as efficiently as possible, i.e. with as few samples as possible. There is of course a tradeoff between accuracy and speed: when far from the minimum, we can determine how to change the parameter even with a very rough estimate of the variance. Near the minimum we need a better estimate of the variance to get further improvements.

I assume that this problem must have already been studied and there must exist standard methods to solve it.

While searching for solutions, I came across the Kiefer-Wolfowitz method. However, this method is for minimizing an expected value, not a variance. Also, there is a big gap between a theoretical description and a practical implementation: one must choose two sequences of numbers, which will have a large effect on the method's performance. I could use some practical advice on how to go about these choices.

Have you thought about using Bayesian Optimisation (BO)? BO employs a Gaussian process to minimise/maximise a function, it is quite useful when your function is noisy. BO will optimise the expected value of your function. We can think of the variance as a function of a parameter $$\theta$$. If the RV is $$X$$ and $$X \sim MyDistribution(\theta)$$ then we will just aim to minimise $$f(\theta) = \log (\text{Var}(X | \theta))$$. In your example, we would use the noisy estimates of the variance you have plotted in the figure on the right.