I found a very mature answer on the Quora and just put it here for people who look for it here:
The Kullback-Leibler divergence has a few nice properties, one of them
being that $πΎπΏ[π;π]$ kind of abhors regions where $π(π₯)$ have
non-null mass and $π(π₯)$ has null mass. This might look like a bug,
but itβs actually a feature in certain situations.
If youβre trying to find approximations for a complex (intractable)
distribution $π(π₯)$ by a (tractable) approximate distribution $π(π₯)$
you want to be absolutely sure that any π₯ that would be very
improbable to be drawn from $π(π₯)$ would also be very improbable to be
drawn from $π(π₯)$. That KL have this property is easily shown: thereβs
a $π(π₯)πππ[π(π₯)/π(π₯)]$ in the integrand. When π(π₯) is small
but $π(π₯)$ is not, thatβs ok. But when $π(π₯)$ is small, this grows very
rapidly if $π(π₯)$ isnβt also small. So, if youβre choosing $π(π₯)$ to
minimize $πΎπΏ[π;π]$, itβs very improbable that $π(π₯)$ will assign a
lot of mass on regions where $π(π₯)$ is near zero.
The Jensen-Shannon divergence donβt have this property. It is well
behaved both when $π(π₯)$ and $π(π₯)$ are small. This means that it wonβt
penalize as much a distribution $π(π₯)$ from which you can sample
values that are impossible in $π(π₯)$.