Analysis of time series with many zero values

This problem is actually about fire detection, but it is strongly analogous to some radioactive decay detection problems. The phenomena being observed is both sporadic and highly variable; thus, a time series will consist of long strings of zeroes interrupted by variable values.

The objective is not merely capturing events (breaks in the zeroes), but quantitative characterization of the events themselves. However, the sensors are limited, and thus will sometimes record zero even if the "reality" is non-zero. For this reason, zeroes must be included when comparing sensors.

Sensor B might be more sensitive than Sensor A, and I would like to be able to describe that statistically. For this analysis, I do not have "truth," but I do have a Sensor C, which is independent of Sensors A&B. Thus my expectation is that better agreement between A/B and C indicates better agreement with "truth." (This may seem shaky, but you'll have to trust me-- I'm on solid ground here, based on what is known from other studies about the sensors).

The problem, then, is how to quantify "better agreement of time series." Correlation is the obvious choice, but will be affected by all those zeroes (which cannot be left out), and of course disproportionately affected by the maximum values. RMSE could also be calculated, but would be strongly weighted toward the behavior of the sensors in the near-zero case.

Q1: What is the best way to apply a logarithmic scaling to non-zero values that will then be combined with zeroes in a time-series analysis?

Q2: What "best practices" can you recommend for a time-series analysis of this type, where behavior at non-zero values is the focus, but zero values dominate and cannot be excluded?

• @assu. The croston() function in the forecast package in R will implement Croston's method for forecasting intermittent demand data. – Rob Hyndman May 27 '11 at 4:07