2
$\begingroup$

Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb R^N$ be fixed. For each $i=1,\ldots,N$, define $e_i(y, a) := \|y-y_i\|^2_2 - a_i$, for $y \in \mathbb R^p$. Now, for positive real numbers $\lambda_1,\ldots,\lambda_N$ and for a standard $k$-dimensional Gaussian random variable $Z$, and for each $i=1,\ldots,N$, consider the transformed random variable $$W_i := \frac{\lambda_i\exp(-e_i(g(Z),a))}{\sum_{j=1}^N\lambda_j\exp(-e_j(g(Z), a))}. $$

Question:

  • (A) What is a "good estimator" for expected value of $W$ ?
  • (B) For which choices of the parameters $\lambda_1,\ldots,\lambda_N$ is the variance of $W_i$ minimized ?

Setting: I want to estimate the expectations of the $W_i$'s to good precision. Indeed, if possible I'd prefer to minimize all the $\operatorname{Var}(W_i)$ at once, i.e solve $$\underset{\lambda_1,\ldots,\lambda_N>0}{\operatorname{minimize}}\;\max_{i} \operatorname{Var}(W_i). $$ I considered the decoupled problems for convenience (presumably much easier).

$\endgroup$
  • $\begingroup$ Notation is better now. What is the application of this problem? $\endgroup$ – Mark L. Stone Jun 27 '18 at 18:41
  • $\begingroup$ Semi-discrete optimal transport. Transporting the pushforward of a Guassian under $g$, unto a sum of Dirac masses. Solution of the above problems is essential for doing gradient updates which don't blow-up. $\endgroup$ – dohmatob Jun 27 '18 at 18:41
  • $\begingroup$ Do you have example instances of g? What are the simplest instances of g of practical; interest? $\endgroup$ – Mark L. Stone Jun 27 '18 at 18:42
  • $\begingroup$ No, $g$ is essentially a black-box (e.g neural network). $\endgroup$ – dohmatob Jun 27 '18 at 18:43
  • 1
    $\begingroup$ EW is gradient, but when you change $\lambda_i$ 's you change EW? Do you then make a correction of some kind to make it all work out? Does this amount to importance sampling? I have no idea what the $y_i$ 's are, so still really don't have a clue what's going on. If you devote yourself to writing out everything more completely, even if it doesn't help you get an answer/suggestion from a reader, it may serve to clarify your own thinking. The whole problem seems perhaps too complicated for a nice forum answer that's really on the money, as opposed to solving the wrong problem. $\endgroup$ – Mark L. Stone Jun 27 '18 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.