I have a dataset (time series data) of measured signal power values from a radio receiver. The data does not originate from a controlled experiment. I have limited knowledge of the underlying processes and distributions.

My observations are the result of one or more processes. In any case, there is a background noise process. Additionally, there can be one or more processes belonging to signals of radio transmitters.

Here are three examples (histograms). In the first case, there is only noise. The second and third case, there is a real mixture of noise and more than one radio transmitter.

only noise example

mixed example 1

mixed example 2

My problem is the following. I want to estimate the fraction of samples that belong to the background noise process. There can be zero to many "non-noise" processes and I have no a priori information about this number.

In general, the sub-populations do not follow the normal distribution (the x-axis in the graphs above is actually logarithmic, note the "dB".) I have no a priori information about their mean and variance. The dataset is huge. Therefore, I am looking for an approach that does not need manual adjustment.

So far, I worked with a simple threshold to separate noise from the rest, but this very inaccurate, especially when there is a signal process with mean very close to the noise. Can you point me to methods which can be applied here?

EDIT: Further reading makes me think that I may want to:

  • Derive a mathematical description of the background noise distribution (curve) from my empirical data.
  • Fit this curve to the sample data.
  • Calculate the ratio of the area under the fitted curve and the total number of samples.

The question is then how to fit the curve? The algorithms I found so far either need the number of components a priori and/or have limitations about the distributions they assume.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.