# Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable

Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb R^N$ be fixed. For each $i=1,\ldots,N$, define $e_i(y, a) := \|y-y_i\|^2_2 - a_i$, for $y \in \mathbb R^p$. Now, for positive real numbers $\lambda_1,\ldots,\lambda_N$ and for a standard $k$-dimensional Gaussian random variable $Z$, and for each $i=1,\ldots,N$, consider the transformed random variable $$W_i := \frac{\lambda_i\exp(-e_i(g(Z),a))}{\sum_{j=1}^N\lambda_j\exp(-e_j(g(Z), a))}.$$

Question:

• (A) What is a "good estimator" for expected value of $W$ ?
• (B) For which choices of the parameters $\lambda_1,\ldots,\lambda_N$ is the variance of $W_i$ minimized ?

Setting: I want to estimate the expectations of the $W_i$'s to good precision. Indeed, if possible I'd prefer to minimize all the $\operatorname{Var}(W_i)$ at once, i.e solve $$\underset{\lambda_1,\ldots,\lambda_N>0}{\operatorname{minimize}}\;\max_{i} \operatorname{Var}(W_i).$$ I considered the decoupled problems for convenience (presumably much easier).

• Notation is better now. What is the application of this problem? Jun 27, 2018 at 18:41
• Semi-discrete optimal transport. Transporting the pushforward of a Guassian under $g$, unto a sum of Dirac masses. Solution of the above problems is essential for doing gradient updates which don't blow-up. Jun 27, 2018 at 18:41
• Do you have example instances of g? What are the simplest instances of g of practical; interest? Jun 27, 2018 at 18:42
• No, $g$ is essentially a black-box (e.g neural network). Jun 27, 2018 at 18:43
• EW is gradient, but when you change $\lambda_i$ 's you change EW? Do you then make a correction of some kind to make it all work out? Does this amount to importance sampling? I have no idea what the $y_i$ 's are, so still really don't have a clue what's going on. If you devote yourself to writing out everything more completely, even if it doesn't help you get an answer/suggestion from a reader, it may serve to clarify your own thinking. The whole problem seems perhaps too complicated for a nice forum answer that's really on the money, as opposed to solving the wrong problem. Jun 27, 2018 at 20:38