# Algorithms for weighted maximum likelihood parameter estimation

What are the computational or algorithmic considerations for weighted maximum likelihood parameter estimation?

That is, I want to get $$\theta^* = \arg\max\limits_\theta \sum_i w_i \log(\mathcal{L}(\theta|x_i))$$ assuming we have a weight $w_i$ for each data point, such that $\sum_i w_i=1$. How is that generally done and are there alternative approaches to finding $\theta^*$?

• Another approach when using SGD optimisation is rejection sampling, with probability proportional to $w_i/w_{\max}$. This is almost never used in practice though.
• Pre-sampling your dataset before applying a standard optimization algorithm is more common. Sample with replacement a new dataset with $w_i/w_{\max}$ proportional sampling. Typically you would take $2n$ to $10n$ samples, where n is the size of your original dataset.