# Entropy of random variables taking real numbers

I need to calculate the entropy of 100 instances of 5 sensor signals in python. The sensor values take real numbers. After doing some literature search, I suppose I need to compute joint differential entropy. For this, I should estimate a multivariate probability density function defining my sensor values. Since I do not have a sound knowledge on information theory, I cannot validate my thoughts. Please guide me how to achieve this.

P.S. I am looking for theoretical suggestions and not related to coding.

• multivariate joint PDF on 100 data points will by garbage – Aksakal Mar 26 '18 at 18:35
• @Aksakal lol. Any suggestions on how to tackle this problem? any alternatives? – Ijjz Mar 26 '18 at 18:39

See Eq.2: $$H(S)=\sum_{i=1}^n \frac {n_i} n\log_2\frac{n_i} n$$ They bin the data, and don't need to estimate the multivariate probability. In their case with 147 sesnors and megabytes of data it would be a garbage joint density, as it would be in your with 5 sensors but only 100 observations.