Why gradient vanishing/exploding is bad? I don't know why gradient vanishing/exploding is a bad thing?
If the gradient of a parameter is small by gradient descent and back propagation, it is the power of Mathematics Rules (Chain Rule) that told the gradient should be this small to get a optimized function value.
So by definition, the gradient value should always be correct no matter how small or large,as long as it is not underflowed or overflowed.
So why we need fight with Math to change gradient calculated by Calculus?
Or are there some bugs in the Chain Rule?
 A: It's nicely explained in Wikipedia and by Basodi, Zhang, and Pan (2020):

Vanishing gradient problem occurs while training artificial neural
networks during backpropagation and can become significant with the
increase of depth of the network. In gradient-based learning methods,
during backpropagation, network weights are updated proportional to
the gradient value (partial derivative of the cost function with
respect to the current weights) after each training iteration (epoch).
Depending on the type of the activation functions and network
architectures, sometimes the gradient value is too small and gets
gradually diminished during backpropagation to the initial layers.
This prevents the network from updating its weights and also sometimes
when the value is too small, the network may be completely stopped
from training (updating weights).

So if you want gradient-based training to find a solution, or at least do it in a finite time, you need to care about it. Also, keep in mind that while real numbers don't have finite precision, numbers as represented on a computer do, so you would inevitably have problems with underflow, completely breaking the computations.
It's not that calculus doesn't work, but about how we use it in gradient descent-based algorithms to train the models.
