I'm working on a HW question in which I'm using the importance sampling method to estimate $E(X)$ where $X$ is distributed as standard Laplace. To do so, I choose my proposal density to be a standard normal. I successfully wrote my code and my estimate is very reasonable. In part of the question I've been asked that "what type of outliers in importance sampling are worrisome". I appreciate if you could guide me on that.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
This is my first answer on stackexchange, so feel free to point out anything I'm doing wrong. Also, I am a student studying this subject so I may make mistakes.
Let's consider the importance weights which are often abbreviated in the literature as P(x)/Q(x). If the proposal density does not have heavy tails while the target density does, then the importance weight will be giving very large importance values to relatively common values in P(x) which are much rarer in Q(x). As Q becomes very small, then the ratio P/Q becomes very large. This will cause outliers to unduly influence the estimate.
Conversely, if the proposal has heavy tails while the target does not, then the estimate will not be grossly distorted, as the ratio P/Q is very small and so this sample would be weighted lightly. This is not optimal, because then we are not incorporating the full value of this sample's information, but at least it is not leading to major distortions of the estimator.
TLDR: I think outliers are worst when the ratio of P/Q is very large because they'll impact the estimator the most. Since your target was Laplace which has heavier tails than the normal, I would not expect that to be an issue with this specific case.