0
$\begingroup$

I want to make a qualitative estimate of the amount of noise in my dataset $X$. For that purpose, I use a certain information estimator $F(X)$ (for example, mutual information between two different variables in a dataset)

I have designed the following procedure:

  1. Generate a fake dataset $Y$ with well-understood properties. Normalize it
  2. Progressively add noise using $G(Y, \alpha) = (1 -\alpha) Y + \alpha N$, where $N \sim \mathcal{N}(0,1)$ and $\alpha$ is sampled within the interval $[0, 1]$
  3. Evaluate $F(G(Y, \alpha))$, plot it as a function of $\alpha$.
  4. Repeat the steps 2,3 for real data to obtain the plot of $F(G(X, \alpha))$
  5. Make a prediction of the amount of noise $\alpha$ in $X$ by evaluating the real data slope $s(X, 0) = \frac{\partial F(G(X, \alpha))}{\partial \alpha} \biggr |_{\alpha=0}$ and solving $s(Y, \alpha) = s(X, 0)$ for $\alpha$. If the plot of $F$ wiggles too much (which it does), I low-pass-filter the plot before evaluating the slopes numerically.

Question:

  • Does this procedure of "blind inference of SNR" have a name?
  • What do people usually do in such scenario?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.