Given a set of positive weights $\{w_i\}_{i=1}^n$, I can normalize them by computing $$W_i = \frac{w_i}{\sum_{j=1}^nw_j}\quad \forall i=1,...,n.$$ Easy enough. But for numerical reasons, it is good to work in logs.


Given the set of unnormalized weights in logs $\{\ln w_i\}_{i=1}^n$, what's the best$^*$ way to compute the set of normalized weights in logs $\{\ln W_i\}_{i=1}^n$?

$^*$ Best meaning most efficient in a runtime and/or numerical sense.

Basic Solution

Given $\{\ln w_i\}_{i=1}^n$, I can recover the $w_i$ by exponentiating the $\ln w_i$, and from there I can compute $$\ln W_i = \ln w_i - \ln\left[\sum_{j=1}^nw_j\right]\quad \forall i = 1,...,n.$$ Can I do better? Is it possible to compute the $\ln W_i$ without exponentiating the $\ln w_i$?

  • 1
    $\begingroup$ Most languages nowadays have some kind of a logsumexp function. I.e. scipy's scipy.special.logsumexp in python does compute the $\log \sum \exp x_i$. It is best to use that if it is available. Otherwise, if your weights could be large before doing the summing the exponentiation and summing you just neetd to subtract the largest weights, so your weigths would all become <=1. $\endgroup$ – sega_sai Nov 9 '18 at 1:43

I don't think there is a way of doing this without exponentiating.

Ho many weights are you using? I can run the full calculation using exponentiation in python for 100,000 weights in about 4ms.

import numpy as np
ln_w = np.random.rand(100000)
ln_W = ln_w - np.log(np.exp(ln_w).sum()) # this line takes 4ms

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.