imagine you have a sample $X_1, X_2, \ldots, X_n$ from a random variable $X$, and another sample $Y_1, Y_2, \ldots, Y_m$ from a random variable $Y$. You know that $Y = \phi(X)$. For concreteness, say $Y = a_0 + a_1 X + a_2 X^2$. How can you estimate $a_0$, $a_1$ and $a_2$ from the samples?

You don't how $X$ or $Y$ are distributed, and your samples do not come in pairs. In fact, $n \neq m$.

I am stuck trying to solve this. Perhaps it is a well-known problem in the statistics community, but I am unable to find anything about it.


  • $\begingroup$ So to be clear, I don't observe pairs of $X$ and $Y$? That is to say $Y_1$ is NOT the transformation of $X_1$? $X_1$ and $Y_1$ are unrelated? $\endgroup$ – Demetri Pananos Jun 10 at 23:35
  • $\begingroup$ No, you don't observe them in pairs. The samples are independent. $\endgroup$ – Jochi Toborochi Jun 11 at 1:51

One method would be constructing a KDE for $Y$ and calculating the likelihood of $\hat Y_i=a_0+a_1X_i+a_2X_i^2$, and then getting maximum likelihood estimate for the parameters $a_0,a_1,a_2$.

More Explanation: Since we don't have corresponding pairs in both datasets, one thing we can do is to estimate the PDF of $Y$, by using some approach. KDE (Kernel Density Estimation) is a useful, non-parametric method for doing this. KDE will gives us an estimate of $f_Y(y)$, i.e. $\hat f_Y(y)$.

Then, we can calculate the likelihood of the $X$ data using: $$L(X|a)=\prod_{i=1}^n \hat f_Y(a_0+a_1x_i+a_2x_i^2)$$

And, find $a_k$ that maximizes this expression. This is mostly a numerical approach.

| cite | improve this answer | |
  • $\begingroup$ Thanks for your answer. Can you elaborate a bit more? $\endgroup$ – Jochi Toborochi Jun 11 at 1:52
  • $\begingroup$ @JochiToborochi I've added more details. $\endgroup$ – gunes Jun 11 at 8:10
  • $\begingroup$ I think the likelihood function of $X$ is $\hat{f}_Y(\phi(x)) / \phi'(x)$, so here $ \hat{f}_Y(\phi(a_0 + a_1 x + a_2 x^2)) / (a_1 + 2 a_2 x)$, no ? $\endgroup$ – Pohoua Jun 11 at 10:01
  • $\begingroup$ Well, not even, since here $\phi$ is not monotonous... but I think the likelihood of $X$ is more complex than just $\hat{f}_Y(\phi(x))$. $\endgroup$ – Pohoua Jun 11 at 10:27
  • 1
    $\begingroup$ Hmm. One point of concern here is that the proposed objective function is trivially maximized by setting $a_1$ and $a_2$ to zero, and $a_0$ to the mode of $\hat{f}_Y$. This would mean that the estimated $a_0$ doesn't depend on $X$, and $a_1$ and $a_2$ don't depend on the data at all. $\endgroup$ – user20160 Jun 11 at 22:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.