16
$\begingroup$

Suppose you fit a linear/logistic regression $g(y) = a_0 + a_1\cdot x_1 + a_2\cdot x_2$, with the aim of an unbiased estimate of $\frac{a_1}{a_2}$. You are very confident that both $a_1$ and $a_2$ are very positive relative to the noise in their estimations.

If you have the joint covariance of $a_1, a_2$, you could calculate, or at least simulate the answer. Are there any better ways, and in real-life problems with a lot of data, how much trouble do you get in for taking the ratio of estimates, or for taking a half-step and assuming the coefficients are independent?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ In logistic regression as described, how do you find an unbiased estimator of $a_0$ or $a_1$? The problem is unrelated with the correlation between the coefficients. $\endgroup$
    – Xi'an
    Commented Sep 18, 2016 at 13:54
  • 6
    $\begingroup$ Something to ponder: What if one or both of the coefficients were zero? $\endgroup$
    – cardinal
    Commented Sep 18, 2016 at 19:59
  • $\begingroup$ Yeah, good point. I'm implicitly assuming that both coefficients are sufficiently positive that there's no danger of noise leading to crossed signs (re: andrewgelman.com/2011/06/21/inference_for_a). I'll edit. $\endgroup$
    – quasi
    Commented Sep 18, 2016 at 22:33
  • 2
    $\begingroup$ How precisely do you estimate $a_1$ and $a_2$ in your regression? Is a consistent estimator with small standard errors sufficient? Is it important that your estimator is unbiased? Would it work for your application to just take $\frac{\hat{a}_1}{\hat{a}_2}$ and calculate the standard-error for that using the delta method and the estimated covariance matrix for $(a_1, a_2)$ from your regression. $\endgroup$
    – Matthew Gunn
    Commented Sep 18, 2016 at 23:07
  • 1
    $\begingroup$ Have you considered Fieller's theorem? Look here: stats.stackexchange.com/questions/16349/… $\endgroup$
    – soakley
    Commented Sep 19, 2016 at 21:36

1 Answer 1

Reset to default
1
$\begingroup$

I would suggest doing error propagation on the variable type and minimize either the error or relative error of $\frac{a_1}{a_2}$. For example, from Strategies for Variance Estimation or Wikipedia

$f = \frac{A}{B}\,$
$\sigma_f^2 \approx f^2 \left[\left(\frac{\sigma_A}{A}\right)^2 + \left(\frac{\sigma_B}{B}\right)^2 - 2\frac{\sigma_{AB}}{AB} \right]$

$\sigma_f \approx \left| f \right| \sqrt{ \left(\frac{\sigma_A}{A}\right)^2 + \left(\frac{\sigma_B}{B}\right)^2 - 2\frac{\sigma_{AB}}{AB} }$

As a guess, you probably want to minimize $(\frac{\sigma_f}{f})^2$. It is important to understand that when one does regression to find a best parameter target, one has forsaken goodness of fit. The fit process will find a best $\frac{A}{B}$, and this is definitively not related to minimizing residuals. This has been done before by taking logarithms of a non-linear fit equation, for which multiple linear applied with a different parameter target and Tikhonov regularization.

The moral of this story is that unless one asks the data to yield the answer that one desires, one will not obtain that answer. And, regression that does not specify the desired answer as a minimization target will not answer the question.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.