From the paper "Campbell and Broderick (2019), Automated Scalable Bayesian Inference via Hilbert Coresets":
We want to create a Bayesian Coreset which is a small weighted subset of our full data, whose weighted log-likelihood approximates the full data log-likelihood. Given $N$ observations, the log-likelihood is
$$ L(\theta) = \sum_{n=1}^{N}L_n \left(\theta \right) $$
where
$$ L_n\left(\theta\right) $$
is the log-likelihood of observation $n$.
The aim of the Bayesian coresets framework is to find a set of nonnegative weights $w:=\left(w_n\right)_{n=1}^{N}$, a small number of which are nonzero, such that the weighted log-likelihood
$$ L(w,\theta) := \sum_{n=1}^{N} w_nL_n\left(\theta\right) \textrm{ satisfies } \lvert L\left(w,\theta\right) - L\left(\theta\right)\rvert \leq \epsilon \lvert L\left(\theta\right)\rvert, \forall \theta \in \Theta $$
To construct a Bayesian coreset we first compute the sensitivity $\sigma_n$ of each data point,
$$ \sigma_n := sup_{\theta\in\Theta}\lvert \frac{L_n\left(\theta\right)}{L\left(\theta\right)} \rvert $$
and then subsample the data set by taking $M$ independent draws with probability proportional to $\sigma_n$ (resulting in a coreset of size $\leq M$) via
$$ \sigma := \sum_{n=1}^{N} \sigma_n $$
$$ \left(M_1,\cdot \cdot \cdot, M_N \right) \sim Multi \left(M,\left( \frac{\sigma_n}{\sigma}\right)_{n=1}^{N} \right) $$
$$ w_n = \frac{\sigma}{\sigma_n} \frac{M_n}{M} $$
Question
Is the "Multi" distribution, a Multinomial distribution with $M$ tries and $N$ possible outcomes for each try? That would mean that an observation could appear multiple times in our subset, is this correct? Also, I don't understand how we came up with $w_n$ as the weights. I observed that this weight is indeed larger for observations which would contribute more to the total maximized log-likelihood ($\sup L_n\left(\theta\right)$ is larger) but what is the thought process behind forming this weight? As I understand it, $\frac{M_n}{M}$ is the proportional frequency of observation $n$ in our subset. Also, $\frac{\sigma}{\sigma_n}$ is the inverse of the proportion of the "contribution" of observation $n$ in the total maximized log-likelihood.
