Quantifying Change in a Histogram Valued Timeseries

Question

I'm attempting to do binary classification where my raw features are collections of histograms that are recorded in a time series. These histograms are scaled to sum to 1.

To be more precise and define some notation, let $H_{t}$ be a histogram at some time $t$ and $H_{t}(i)$ it's value in the $i$th bucket (here the $i$th bucket is produced by binning non-negative integer values). I have such an $H_{t}$ for each one of my data points (so I could write $H^k_t$ to represent the histogram associated with data point $k$ at time $t$).

It is somewhat hopeless to use the histograms as features as what seems to be most relevant to my classification is the change that is occurring in the histograms as time changes. My current strategy to quantify this change is to, for each $i$, fix bucket $i$ and then preform a discrete wavelet transform along $t$ using the haar wavelet ($t$ takes a power of 2 many values so the DWT is easy to apply).

While this approach works somewhat well it is constrained by it's inability to pick up simultaneous and general changes in the histogram; for instance it is heuristically probable in my dataset that a decrease in the value of bucket $i=2,3$ is 'bad' unless it is accompanied by an increase in the values of some larger buckets, that is to say that the histogram is shifting strictly right.

My question is then: are there are any general techniques/strategies that describe the change of histograms over time that capture these more general changes? I should note that the ranges of $i$ and $t$ are fairly small, less than 10 each.

Answer 1

I have a suggestion for a slightly different strategy. Depending on whether your data is discrete or continuous, a histogram is essentially an estimator of the probability density or mass function. If $P^k_t$ and $P^k_{t+1}$ are the probability distributions of point $k$ at time $t$ and $t+1$, respectively, you could try to compute Kullback-Liebler divergence between $P^k_t$ and $P^k_{t+1}$ as a measure of dissimilarity between these two distributions. Your value for the density/mass at point $i$ will be the value of the histogram bin. You could then produce a time-series of these divergence scores and investigate any spikes. Changing the frequency of $t$ may be a good idea; i.e. you may not observe any changes between daily observations but over a week or a month results could be different

Kullback-Liebler divergence is actually one type of $f$-divergence so there are different measures you could use. These tools are available in many statistical packages.