4
$\begingroup$

I guess I come up with a classic question, but I failed to find any useful solutions by far. My question is about the following model $$y=x+n$$ where $x$ is a hidden random variable that cannot be observed, $n$ is the random variable of white noise, namely $n$ follows a known Gaussian distribution of variance $\sigma^2$ ($f_n= {\cal{N}}(0,\sigma^2)$), and $y$ is a random variable that we can observe.

Suppose we know the distribution of $y$ as $f_y$, and I wonder how to find the pdf of $x$. Theoretically, it seems to be equivalent of the sum of two dependent random variables i.e. $x = y-n$, but in this question how $y$ and $n$ are correlated is unknown. Is there any existing solution?

Thanks

$\endgroup$
1
  • $\begingroup$ Do you know if $x$ and $n$ can be treated as independent random variables? If so, the characteristic function $\Psi_y$ of $y$ is the product of the characteristic functions $\Psi_x$ and $\Psi_n$ of $x$ and $n$ respectively, and so, since $\Psi_n$ is nonzero everywhere, you get the characteristic function $\Psi_x$ as being just $\Psi_y/\Psi_n$ from which the pdf of $y$ can be obtained. $\endgroup$ – Dilip Sarwate Oct 28 '13 at 23:32
1
$\begingroup$

SCENARIO 1: $X$ and $N$ independent.

Re-iterating and expanding a bit on Pof. Sarwate comment, if we can assume that $X$ and $N$ are independent, then the characteristic function of $Y$ is equal to the product of the characteristic functions of $X$ and $N$. Since we know the density of $Y$ (or accept an estimation of it through the available observations), we can formulate its characteristic function also.

$$\Psi_Y(t) = \Psi_X(t) \cdot \Psi_N(t) \Rightarrow \Psi_X(t) = \frac {\Psi_Y(t)}{\Psi_N(t)}$$

where $$\Psi_Z(t) = E\left(\exp{\left\{izt\right\}}\right)$$

with $i$ being the imaginary unit.

Of course, finding the characteristic function of the distribution of $X$ does not "guarantee" that we will recognize a "known" distribution to which corresponds a readily available probability density function. And this is because, while $N$ is a normal random variable, it may be the case that $Y$ may not follow such a "known" distribution whose characteristic function has already been derived. Or even if we have available the characteristic function of $Y$, still, the ratio of characteristic functions above may not look familiar. Nevertheless, the characteristic function... characterizes the distribution, and we can do a lot with it, once we have it available.

SCENARIO 2: $X$ and $N$ not-independent

This essentially is not one case but infinitely many. For example, since $Y$ is observable, presumably it is repeatedly observable. So are we talking about dependence for the same value of the index, or also for dependence across the index? Etc

In any case, the first thing we should think of doing, since $N$ is normal, is to examine and test whether the empirical distribution of $Y$ looks also as being a Normal. If it does, then I believe we are excused if we attempt to model $X$ as a Normal r.v. also.

In such a case, we already know the mean of $X$, $E(X) = E(Y)$ (and so we can etimate it from the sample of relaizations of $Y$), and we are after its variance in order to completely characterize the distribution. We have

$$\operatorname{Var}(Y) = \operatorname{Var}(X) + \operatorname{Var}(N) +2\operatorname{Cov}(X,N) = \sigma^2_x + \sigma^2_n + 2\rho_{x,n}\sigma_n\sigma_x >0$$

The variance of $N$ is known, and the variance of $Y$ can be estimated from the sample. Since the correlation coefficient is bounded in $[-1,1]$ we could obtain (estimated) bounds for $\sigma^2_x$. But we may be able to do better.

We can write the above relation as a quadratic polynomial in $\sigma_x$,

$$\sigma^2_x + 2\rho_{x,n}\sigma_n\sigma_x +(\sigma^2_n- \hat\sigma^2_y) =0$$

Its discriminant will be

$$\Delta_x = 4\rho_{x,n}^2\sigma^2_n - 4(\sigma^2_n- \hat\sigma^2_y)$$

and so the roots of the quadratic will be

$${\sigma_x}_{1,2} = \frac {-2\rho_{x,n}\sigma_n \pm \sqrt {4\rho_{x,n}^2\sigma^2_n - 4(\sigma^2_n- \hat\sigma^2_y)}}{2}$$

$$=-\rho_{x,n}\sigma_n \pm \sqrt {\hat\sigma^2_y -(1-\rho_{x,n}^2)\sigma^2_n}$$

The term under the square root must be real so we obtain the restriction

$$\hat\sigma^2_y -(1-\rho_{x,n}^2)\sigma^2_n \geq 0 \Rightarrow \rho_{x,n}^2 \geq 1-\frac{\hat\sigma^2_y}{\sigma^2_n}$$

So if it so happens that $\hat\sigma^2_y < \sigma^2_n$ then we obtain an effective lower bound on the absolute value of the correlation coefficient, reducing its possible values.

Moreover, we must have

$$\sigma_x >0 \Rightarrow -\rho_{x,n}\sigma_n \pm \sqrt {\rho_{x,n}^2\sigma^2_n -(\sigma^2_n-\hat\sigma^2_y) } >0$$

From this required inequality it is easy to realize that $\hat\sigma^2_y < \sigma^2_n \Rightarrow \rho_{x,n}< 0 $ and combining results

$$\hat\sigma^2_y < \sigma^2_n \Rightarrow \rho_{x,n} < -\sqrt {1-\frac{\hat\sigma^2_y}{\sigma^2_n}}$$

On the other hand, if $\hat\sigma^2_y > \sigma^2_n$ no restrictions on the possible values of the correlation coefficient arise.

(End of Episode II)

$\endgroup$
6
  • $\begingroup$ The question asserts that the correlation between the variables is unknown, which means we cannot assume independence. $\endgroup$ – whuber Jul 22 '14 at 15:04
  • $\begingroup$ @whuber The text of the question does not explicitly exclude the case of $X$ being independent of $N$. So I don't see why not providing an answer for the case of independence. Obviously, if $X$ and $N$ are stochastically dependent, another treatment is called for. $\endgroup$ – Alecos Papadopoulos Jul 22 '14 at 15:17
  • $\begingroup$ Even though this very special case has not been specifically ruled out, the OP has made it clear that we cannot assume it is the case. Doing so (at least without appropriate checks and diagnostics) would therefore be an error. $\endgroup$ – whuber Jul 22 '14 at 15:22
  • $\begingroup$ @whuber But this is the OP's work (to check and diagnose), isn't it? We do not posses the data, neither any specific information about the whole situation, the OP does. And my answer explicitly starts with "If we can assume that $X$ and $N$ are independent..." $\endgroup$ – Alecos Papadopoulos Jul 22 '14 at 15:40
  • $\begingroup$ There's nothing wrong with this answer, but it seems a little less than fully helpful. The entire point of the question, as I read it, asks us to focus on to what extent can we solve this problem when we cannot make such a simplifying assumption? $\endgroup$ – whuber Jul 22 '14 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.