I have a function $f$ which maps real numbers between $0$ and $10$ to $\mathbb{R_+}^{1000}$. For example, $f(4.0)$ returns a set of thousand positive numbers like $1.0, 1.4, 4.3, 0.5$, etc. Theoretically the elements in the resulting set can take any positive value, but most are small.

To begin with, I would like to find the value $x$ that minimizes the mean of $f(x)$. This is not difficult in my case; it helps that $g(x) = \text{mean}(f(x))$ is a continuous function. However, the mean is not so good if the variance of the resulting set is high. For example, $\{10, 0, 10, 0, \dots \}$ is less optimal than $\{ 5.1, 5.1, \dots\}$. Therefore I would like to minimize the standard deviation as well. This is a problem since the number that minimizes the mean of $f$ and the value that minimizes the standard deviation of $f$ may be different. Therefore I may need to define some formula involving the mean and standard deviation which I would then seek to minimize, or try to see if the there exists a local minima of the standard deviation in the neighbourhood of the global minima of the mean, or just to to put some constraint on the standard deviation, etc. My question is: does there exist a standard procedure or a framework for solving this problem which would let me define as few hyperparameters as possible?

  • 2
    $\begingroup$ For multi-objective optimization how to proceed really depends on what you want to achieve, so I would say there is not necessarily a standard. In your case you say the mean is continuous. Is the standard deviation also continuous? And how dense are minima of $g[x]$ on $x\in[0,10]$? In other words can you try plotting the mean and standard deviation as a function of $x$ to get an idea of the spatial distribution of minima in each? (Is evaluating $f$ very expensive?) $\endgroup$ – GeoMatt22 Jan 6 '17 at 18:11
  • $\begingroup$ @GeoMatt22 Thank you. Yes the standard deviation is also continuous. What I want to achieve is as low mean as possible and to a lesser degree a low standard deviation, but quantifying that in exact terms makes me take a subjective position which I would have liked to avoid. I think I can do what you are suggesting. I understand that it would give me some useful information, but how exactly would it help me? $\endgroup$ – Sid Jan 6 '17 at 18:45
  • $\begingroup$ Since you have a 1D domain, it is easy to visualize the statistics as a function of $x$ (or, at a more sophisticated level, to do surrogate modeling/design of computational experiments). So you could sample the population of local minima of the mean, and see how the standard deviation co-varies with the mean, over different parts of the domain. Essentially your problem would then be a choice over a finite set of $x_\min$ points, rather than a continuous interval in $x$. $\endgroup$ – GeoMatt22 Jan 6 '17 at 20:03
  • 3
    $\begingroup$ You can try to calculate a Pareto frontier, that is, for each given $\sigma$, fond the optimal mean given that standard devoation is at most $\sigma$. Plot that as a function, might help to find an compromise solution. $\endgroup$ – kjetil b halvorsen Apr 14 '17 at 1:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.