# Can anyone suggest minimum and maximum values that the relative entropy/KL divergence can range when computed from two time-series sequences?

Can anyone suggest minimum and maximum values that the relative entropy/KL divergence can range when computed from two time-series sequences?

I have calculated the relative entropy for two time-series sequences and it gives me greater than zero values. From the literature, I came to know that relative entropy will never have less than zero value. Moreover, if the value is zero then it suggests that these two time-series sequences are identical.

• My question is how do I interpret the relative entropy values? Like I want to understand that how much one probability distribution (e.g. first time series) differs from another probability distribution (e.g. second time-series). In this case, I have applied relative entropy on these two sequences. Hope this will clear the question. May 10, 2021 at 11:14

As an extremely simple example, suppose we are looking at two discrete distributions with only two outcomes, where $$P(0)=P(1)=\frac{1}{2}, \quad Q(0)=q, \quad Q(1)=1-q \quad\text{for }0 Then the entropy is $$\sum_{x\in\mathcal{X}} P(x)\log\frac{P(x)}{Q(x)} = \frac{1}{2}\bigg(\log\frac{1}{2q}+\log\frac{1}{2(1-q)}\bigg) = \frac{1}{2}\big(-2\log 2-\log q-\log(1-q)\big).$$
And $$-\log q-\log(1-q)$$ is unbounded for $$q$$ near $$0$$ or $$1$$:
qq <- seq(.001,.999,by=.001)